[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:39 dist-test-slave-sb2z krb5kdc[27967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:39 dist-test-slave-sb2z krb5kdc[27967](info): set up 2 sockets
Aug 06 00:53:39 dist-test-slave-sb2z krb5kdc[27967](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 00:53:40.486070 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.393s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:53:40 dist-test-slave-sb2z krb5kdc[27967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:53:40.632715 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:39191
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:45191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:39191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:40.750833 27989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:40.751144 27989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:40.751188 27989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:40.755504 27989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:40.755630 27989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:40.755651 27989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:40.755677 27989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:40.755693 27989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:40.761439 27989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:45191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:39191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:39191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:40.762631 27989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:40.763715 27989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:40.774453 27995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:40.774453 27997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:40.774617 27994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:40.774513 27989 server_base.cc:1034] running on GCE node
I20260806 00:53:40.775511 27989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:40.776896 27989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:40.778142 27989 hybrid_clock.cc:648] HybridClock initialized: now 1785977620778105 us; error 53 us; skew 500 ppm
Aug 06 00:53:40 dist-test-slave-sb2z krb5kdc[27967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977620, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:40.786208 27989 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:53:40.787356 27989 webserver.cc:460] Webserver started at http://127.27.77.254:37647/ using document root <none> and password file <none>
I20260806 00:53:40.788053 27989 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:40.788156 27989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:40.788414 27989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:40.790726 27989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "c18e51f01c674bdb8a895562a3a59289"
format_stamp: "Formatted at 2026-08-06 00:53:40 on dist-test-slave-sb2z"
server_key: "f526b2eeef21129ae52911c0c00fbb1f"
server_key_iv: "e6e18a2f9423f89901a64fe74e46de87"
server_key_version: "encryptionkey@0"
I20260806 00:53:40.791345 27989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "c18e51f01c674bdb8a895562a3a59289"
format_stamp: "Formatted at 2026-08-06 00:53:40 on dist-test-slave-sb2z"
server_key: "f526b2eeef21129ae52911c0c00fbb1f"
server_key_iv: "e6e18a2f9423f89901a64fe74e46de87"
server_key_version: "encryptionkey@0"
I20260806 00:53:40.795348 27989 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 00:53:40.797950 28007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:40.799310 27989 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:53:40.799472 27989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "c18e51f01c674bdb8a895562a3a59289"
format_stamp: "Formatted at 2026-08-06 00:53:40 on dist-test-slave-sb2z"
server_key: "f526b2eeef21129ae52911c0c00fbb1f"
server_key_iv: "e6e18a2f9423f89901a64fe74e46de87"
server_key_version: "encryptionkey@0"
I20260806 00:53:40.799659 27989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:40.826097 27989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:40.830672 27989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:40.830878 27989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:40.840476 27989 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:39191
I20260806 00:53:40.840606 28072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:39191 every 8 connection(s)
I20260806 00:53:40.841660 27989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:40.844813 28074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:40.851797 28074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289: Bootstrap starting.
I20260806 00:53:40.851826 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 27989
I20260806 00:53:40.852002 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:53:40.853415 27959 external_mini_cluster.cc:1428] Setting key df0c98c4c50b38b0cf033beaea259135
I20260806 00:53:40.854844 28074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:40.855849 28074 log.cc:824] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:40.858207 28074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289: No bootstrap required, opened a new log
I20260806 00:53:40.863229 28074 raft_consensus.cc:348] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18e51f01c674bdb8a895562a3a59289" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39191 } }
I20260806 00:53:40.863596 28074 raft_consensus.cc:374] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:40.863669 28074 raft_consensus.cc:729] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c18e51f01c674bdb8a895562a3a59289, State: Initialized, Role: FOLLOWER
I20260806 00:53:40.864183 28074 consensus_queue.cc:260] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18e51f01c674bdb8a895562a3a59289" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39191 } }
I20260806 00:53:40.864336 28074 raft_consensus.cc:388] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:40.864425 28074 raft_consensus.cc:482] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:40.864538 28074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:40.865670 28074 raft_consensus.cc:504] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18e51f01c674bdb8a895562a3a59289" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39191 } }
I20260806 00:53:40.866051 28074 leader_election.cc:302] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c18e51f01c674bdb8a895562a3a59289; no voters:
I20260806 00:53:40.866441 28074 leader_election.cc:288] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:40.866681 28079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:40.867125 28079 raft_consensus.cc:686] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [term 1 LEADER]: Becoming Leader. State: Replica: c18e51f01c674bdb8a895562a3a59289, State: Running, Role: LEADER
I20260806 00:53:40.867486 28079 consensus_queue.cc:237] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18e51f01c674bdb8a895562a3a59289" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39191 } }
I20260806 00:53:40.868077 28074 sys_catalog.cc:564] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:40.869431 28080 sys_catalog.cc:455] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c18e51f01c674bdb8a895562a3a59289" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18e51f01c674bdb8a895562a3a59289" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39191 } } }
I20260806 00:53:40.869581 28080 sys_catalog.cc:458] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:40.870000 28082 sys_catalog.cc:455] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c18e51f01c674bdb8a895562a3a59289. Latest consensus state: current_term: 1 leader_uuid: "c18e51f01c674bdb8a895562a3a59289" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c18e51f01c674bdb8a895562a3a59289" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39191 } } }
I20260806 00:53:40.870139 28082 sys_catalog.cc:458] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289 [sys.catalog]: This master's current role is: LEADER
W20260806 00:53:40.874929 28100 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:53:40.875066 28100 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:53:40.875171 28095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:40.876039 28095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 00:53:40 dist-test-slave-sb2z krb5kdc[27967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:40.882627 28095 catalog_manager.cc:1347] Generated new cluster ID: 38e77c282f93486e96eefa938151bef6
I20260806 00:53:40.882707 28095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:40.885592 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:40.857107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57844 (local address 127.27.77.254:39191)
0806 00:53:40.858594 (+ 1487us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:40.858616 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:40.859084 (+ 468us) server_negotiation.cc:407] Connection header received
0806 00:53:40.862648 (+ 3564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:40.862670 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:40.863116 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:40.863543 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:40.866984 (+ 3441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:40.868801 (+ 1817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:40.870757 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:40.871573 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:40.871757 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:40.877648 (+ 5891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:40.877655 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:40.877672 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:40.877708 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:40.879793 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:40.880884 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:40.880889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:40.880893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:40.881023 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:40.881518 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:40.881522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:40.881524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:40.882914 (+ 1390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:40.883209 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:40.884357 (+ 1148us) server_negotiation.cc:299] Negotiation successful
0806 00:53:40.884677 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1313,"thread_start_us":152,"threads_started":1}
I20260806 00:53:40.892052 28095 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:40.893452 28095 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:40.902016 28095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c18e51f01c674bdb8a895562a3a59289: Generated new TSK 0
I20260806 00:53:40.902840 28095 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:53:40.943095 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39191
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:45191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:41.073647 28106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:41.073940 28106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:41.073987 28106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:41.078234 28106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:41.078318 28106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:41.078382 28106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:53:41.084245 28106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:45191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39191
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:41.085474 28106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:41.086629 28106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:53:41.097720 28106 server_base.cc:1034] running on GCE node
W20260806 00:53:41.097615 28114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:41.097589 28116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:41.097589 28113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:41.098351 28106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:41.099094 28106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:41.100276 28106 hybrid_clock.cc:648] HybridClock initialized: now 1785977621100240 us; error 50 us; skew 500 ppm
Aug 06 00:53:41 dist-test-slave-sb2z krb5kdc[27967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977621, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:41.103653 28106 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:53:41.104784 28106 webserver.cc:460] Webserver started at http://127.27.77.193:35107/ using document root <none> and password file <none>
I20260806 00:53:41.105424 28106 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:41.105494 28106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:41.105746 28106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:41.108124 28106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "1e4ae0fbfba149e99d2447cf4d543256"
format_stamp: "Formatted at 2026-08-06 00:53:41 on dist-test-slave-sb2z"
server_key: "61ab22d10aae78f88cdc5fcd730e75b9"
server_key_iv: "a9ac0eba7be7d8939dcc10ef57bb5f92"
server_key_version: "encryptionkey@0"
I20260806 00:53:41.108764 28106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "1e4ae0fbfba149e99d2447cf4d543256"
format_stamp: "Formatted at 2026-08-06 00:53:41 on dist-test-slave-sb2z"
server_key: "61ab22d10aae78f88cdc5fcd730e75b9"
server_key_iv: "a9ac0eba7be7d8939dcc10ef57bb5f92"
server_key_version: "encryptionkey@0"
I20260806 00:53:41.113148 28106 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:53:41.115922 28125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:41.117256 28106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:53:41.117370 28106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "1e4ae0fbfba149e99d2447cf4d543256"
format_stamp: "Formatted at 2026-08-06 00:53:41 on dist-test-slave-sb2z"
server_key: "61ab22d10aae78f88cdc5fcd730e75b9"
server_key_iv: "a9ac0eba7be7d8939dcc10ef57bb5f92"
server_key_version: "encryptionkey@0"
I20260806 00:53:41.117520 28106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:41.133669 28106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:41.137866 28106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:41.138142 28106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:41.138900 28106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:53:41.139954 28106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:53:41.140010 28106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:41.140086 28106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:53:41.140123 28106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:41.151304 28106 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:36285
I20260806 00:53:41.151325 28279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:36285 every 8 connection(s)
I20260806 00:53:41.152431 28106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:53:41.161952 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 28106
I20260806 00:53:41.162158 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:53:41.162484 27959 external_mini_cluster.cc:1428] Setting key 4b8108fb208452d2a6f675e759245f93
Aug 06 00:53:41 dist-test-slave-sb2z krb5kdc[27967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977621, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:41.168349 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:41.154823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:46683 (local address 127.27.77.254:39191)
0806 00:53:41.154990 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:41.154994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:41.156268 (+ 1274us) server_negotiation.cc:407] Connection header received
0806 00:53:41.157171 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:41.157173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:41.157230 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:41.157312 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:41.158604 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.159142 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.160470 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.160707 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.160806 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:41.163811 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:41.163815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:41.163817 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:41.163845 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:41.165704 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:41.166306 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:41.166313 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:41.166318 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:41.166430 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:41.166892 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:41.166895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:41.166896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:41.167110 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:41.167193 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:41.167949 (+ 756us) server_negotiation.cc:299] Negotiation successful
0806 00:53:41.168125 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 00:53:41.169574 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:41.155266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:39191 (local address 127.27.77.193:46683)
0806 00:53:41.156060 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:41.156148 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:41.156968 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:41.157489 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:41.157495 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:41.157879 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:41.158417 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:41.158430 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.159274 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.159276 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:41.160320 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:41.160331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.160873 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.160876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:41.161055 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:41.161671 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:41.161688 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:41.163612 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:41.165857 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:41.165863 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:41.165878 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:41.166181 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:41.166541 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:41.166547 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:41.166551 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:41.166773 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:41.167224 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:41.167233 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:41.167642 (+ 409us) client_negotiation.cc:769] Sending connection context
0806 00:53:41.167882 (+ 240us) client_negotiation.cc:240] Negotiation successful
0806 00:53:41.168358 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":131,"threads_started":1}
I20260806 00:53:41.171357 28280 heartbeater.cc:347] Connected to a master server at 127.27.77.254:39191
I20260806 00:53:41.171795 28280 heartbeater.cc:464] Registering TS with master...
I20260806 00:53:41.172405 28280 heartbeater.cc:511] Master 127.27.77.254:39191 requested a full tablet report, sending...
I20260806 00:53:41.174203 28033 ts_manager.cc:194] Registered new tserver with Master: 1e4ae0fbfba149e99d2447cf4d543256 (127.27.77.193:36285)
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
I20260806 00:53:41.175467 28033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:46683
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:53:41.200095 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39191
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:45191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:41.327785 28287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:41.328087 28287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:41.328174 28287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:41.332993 28287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:41.333073 28287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:41.333182 28287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:53:41.338958 28287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:45191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39191
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:41.340183 28287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:41.341269 28287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:41.351845 28296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:41.351845 28294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:41.351866 28292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:41.352492 28287 server_base.cc:1034] running on GCE node
I20260806 00:53:41.352883 28287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:41.353627 28287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:41.354810 28287 hybrid_clock.cc:648] HybridClock initialized: now 1785977621354785 us; error 37 us; skew 500 ppm
Aug 06 00:53:41 dist-test-slave-sb2z krb5kdc[27967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977621, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:41.358278 28287 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:53:41.359239 28287 webserver.cc:460] Webserver started at http://127.27.77.194:43257/ using document root <none> and password file <none>
I20260806 00:53:41.359910 28287 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:41.359982 28287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:41.360190 28287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:41.362445 28287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "fe5d0d8cd89448cb880d069200fccc71"
format_stamp: "Formatted at 2026-08-06 00:53:41 on dist-test-slave-sb2z"
server_key: "c0f88c8beb701e321e7f4b978faab12c"
server_key_iv: "d09a8e951db17aa3165872f2d94b4aea"
server_key_version: "encryptionkey@0"
I20260806 00:53:41.363010 28287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "fe5d0d8cd89448cb880d069200fccc71"
format_stamp: "Formatted at 2026-08-06 00:53:41 on dist-test-slave-sb2z"
server_key: "c0f88c8beb701e321e7f4b978faab12c"
server_key_iv: "d09a8e951db17aa3165872f2d94b4aea"
server_key_version: "encryptionkey@0"
I20260806 00:53:41.367259 28287 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 00:53:41.370631 28306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:41.372063 28287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 00:53:41.372177 28287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "fe5d0d8cd89448cb880d069200fccc71"
format_stamp: "Formatted at 2026-08-06 00:53:41 on dist-test-slave-sb2z"
server_key: "c0f88c8beb701e321e7f4b978faab12c"
server_key_iv: "d09a8e951db17aa3165872f2d94b4aea"
server_key_version: "encryptionkey@0"
I20260806 00:53:41.372282 28287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:41.407461 28287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:41.410665 28287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:41.410842 28287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:41.411432 28287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:53:41.412365 28287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:53:41.412420 28287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:41.412462 28287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:53:41.412477 28287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:41.423188 28287 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:35653
I20260806 00:53:41.423255 28465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:35653 every 8 connection(s)
I20260806 00:53:41.424254 28287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:53:41.429055 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 28287
I20260806 00:53:41.429150 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:53:41.429402 27959 external_mini_cluster.cc:1428] Setting key ead2a6a1c15a3418345561bda5809b06
Aug 06 00:53:41 dist-test-slave-sb2z krb5kdc[27967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977621, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:41.439466 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:41.426301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:38207 (local address 127.27.77.254:39191)
0806 00:53:41.426451 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:41.426455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:41.427577 (+ 1122us) server_negotiation.cc:407] Connection header received
0806 00:53:41.428513 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:41.428515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:41.428567 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:41.428649 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:41.430008 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.430566 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.431820 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.432084 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.432216 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:41.435410 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:41.435414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:41.435416 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:41.435449 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:41.437170 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:41.437759 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:41.437763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:41.437765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:41.437854 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:41.438238 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:41.438242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:41.438243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:41.438456 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:41.438571 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:41.439149 (+ 578us) server_negotiation.cc:299] Negotiation successful
0806 00:53:41.439291 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
I20260806 00:53:41.440514 28469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:41.426703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:39191 (local address 127.27.77.194:38207)
0806 00:53:41.427383 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:41.427426 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:41.428314 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:41.428823 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:41.428829 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:41.429301 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:41.429852 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:41.429866 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.430710 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.430713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:41.431694 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:41.431705 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.432260 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.432264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:41.432438 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:41.433132 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:41.433165 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:41.435214 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:41.437314 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:41.437320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:41.437339 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:41.437640 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:41.437955 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:41.437958 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:41.437961 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:41.438125 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:41.438587 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:41.438597 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:41.438893 (+ 296us) client_negotiation.cc:769] Sending connection context
0806 00:53:41.439105 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 00:53:41.439490 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":122,"threads_started":1}
I20260806 00:53:41.442116 28467 heartbeater.cc:347] Connected to a master server at 127.27.77.254:39191
I20260806 00:53:41.442492 28467 heartbeater.cc:464] Registering TS with master...
I20260806 00:53:41.443130 28467 heartbeater.cc:511] Master 127.27.77.254:39191 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
I20260806 00:53:41.444494 28033 ts_manager.cc:194] Registered new tserver with Master: fe5d0d8cd89448cb880d069200fccc71 (127.27.77.194:35653)
I20260806 00:53:41.445387 28033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:38207
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:53:41.463455 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39191
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:45191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:41.585371 28472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:41.585671 28472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:41.585721 28472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:41.589896 28472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:41.589973 28472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:41.590078 28472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:53:41.595824 28472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:45191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39191
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:41.597090 28472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:41.598229 28472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:41.612052 28479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:41.612056 28483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:41.612129 28472 server_base.cc:1034] running on GCE node
W20260806 00:53:41.612056 28480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:41.612785 28472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:41.613446 28472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:41.614668 28472 hybrid_clock.cc:648] HybridClock initialized: now 1785977621614650 us; error 31 us; skew 500 ppm
Aug 06 00:53:41 dist-test-slave-sb2z krb5kdc[27967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977621, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:41.617724 28472 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:53:41.618655 28472 webserver.cc:460] Webserver started at http://127.27.77.195:35549/ using document root <none> and password file <none>
I20260806 00:53:41.619251 28472 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:41.619349 28472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:41.619654 28472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:41.621951 28472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "65a7d0a0bf3f420e8bd98da83bf00ff0"
format_stamp: "Formatted at 2026-08-06 00:53:41 on dist-test-slave-sb2z"
server_key: "2a488cff20a50b938cc27d4854e5818b"
server_key_iv: "52be035da5ecb2a9849b9704f4821db1"
server_key_version: "encryptionkey@0"
I20260806 00:53:41.622562 28472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "65a7d0a0bf3f420e8bd98da83bf00ff0"
format_stamp: "Formatted at 2026-08-06 00:53:41 on dist-test-slave-sb2z"
server_key: "2a488cff20a50b938cc27d4854e5818b"
server_key_iv: "52be035da5ecb2a9849b9704f4821db1"
server_key_version: "encryptionkey@0"
I20260806 00:53:41.626488 28472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:53:41.629362 28492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:41.630733 28472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:53:41.630889 28472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "65a7d0a0bf3f420e8bd98da83bf00ff0"
format_stamp: "Formatted at 2026-08-06 00:53:41 on dist-test-slave-sb2z"
server_key: "2a488cff20a50b938cc27d4854e5818b"
server_key_iv: "52be035da5ecb2a9849b9704f4821db1"
server_key_version: "encryptionkey@0"
I20260806 00:53:41.631018 28472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:41.643631 28472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:41.646838 28472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:41.647061 28472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:41.647725 28472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:53:41.648597 28472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:53:41.648670 28472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:41.648743 28472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:53:41.648787 28472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:41.659287 28472 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:44479
I20260806 00:53:41.659309 28642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:44479 every 8 connection(s)
I20260806 00:53:41.660306 28472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:53:41.661846 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 28472
I20260806 00:53:41.661988 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:53:41.662310 27959 external_mini_cluster.cc:1428] Setting key 0062a6d50a8f21b9a6e857627ecfaba1
Aug 06 00:53:41 dist-test-slave-sb2z krb5kdc[27967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977621, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:41.675038 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:41.662374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:45151 (local address 127.27.77.254:39191)
0806 00:53:41.662540 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:41.662545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:41.663761 (+ 1216us) server_negotiation.cc:407] Connection header received
0806 00:53:41.664630 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:41.664633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:41.664691 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:41.664779 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:41.665977 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.666507 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.667725 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.668034 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.668145 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:41.671073 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:41.671076 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:41.671078 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:41.671134 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:41.672693 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:41.673175 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:41.673178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:41.673182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:41.673264 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:41.673588 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:41.673590 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:41.673591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:41.673845 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:41.673961 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:41.674743 (+ 782us) server_negotiation.cc:299] Negotiation successful
0806 00:53:41.674901 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 00:53:41.676084 28646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:41.662789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:39191 (local address 127.27.77.195:45151)
0806 00:53:41.663583 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:41.663639 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:41.664425 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:41.664976 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:41.664983 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:41.665355 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:41.665826 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:41.665840 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.666649 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.666653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:41.667592 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:41.667604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.668202 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.668206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:41.668378 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:41.669101 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:41.669117 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:41.670903 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:41.672803 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:41.672807 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:41.672822 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:41.673085 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:41.673347 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:41.673349 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:41.673350 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:41.673489 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:41.673957 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:41.673965 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:41.674398 (+ 433us) client_negotiation.cc:769] Sending connection context
0806 00:53:41.674658 (+ 260us) client_negotiation.cc:240] Negotiation successful
0806 00:53:41.675021 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":152,"threads_started":1}
I20260806 00:53:41.677448 28643 heartbeater.cc:347] Connected to a master server at 127.27.77.254:39191
I20260806 00:53:41.677706 28643 heartbeater.cc:464] Registering TS with master...
I20260806 00:53:41.678231 28643 heartbeater.cc:511] Master 127.27.77.254:39191 requested a full tablet report, sending...
I20260806 00:53:41.679361 28033 ts_manager.cc:194] Registered new tserver with Master: 65a7d0a0bf3f420e8bd98da83bf00ff0 (127.27.77.195:44479)
I20260806 00:53:41.680024 28033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:45151
I20260806 00:53:41.689268 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 00:53:41 dist-test-slave-sb2z krb5kdc[27967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977621, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 00:53:41 dist-test-slave-sb2z krb5kdc[27967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977621, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.77.193@KRBTEST.COM
I20260806 00:53:41.719265 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:41.709000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40432 (local address 127.27.77.193:36285)
0806 00:53:41.709336 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:41.709343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:41.709371 (+ 28us) server_negotiation.cc:407] Connection header received
0806 00:53:41.709514 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:41.709520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:41.709740 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:41.709870 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:41.710613 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.711423 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.712328 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.712747 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.712844 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:41.715587 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:41.715601 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:41.715615 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:41.715644 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:41.717501 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:41.717949 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:41.717954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:41.717955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:41.718042 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:41.718361 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:41.718363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:41.718364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:41.718627 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:41.718787 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:41.718958 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 00:53:41.719116 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":105,"threads_started":1}
W20260806 00:53:41.720494 28160 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40432
Aug 06 00:53:41 dist-test-slave-sb2z krb5kdc[27967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 00:53:41 dist-test-slave-sb2z krb5kdc[27967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.193@KRBTEST.COM
I20260806 00:53:41.750696 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:41.740823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40438 (local address 127.27.77.193:36285)
0806 00:53:41.741101 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:41.741105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:41.741122 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:53:41.741310 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:41.741312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:41.741365 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:41.741446 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:41.742284 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.742800 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.743807 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:41.744112 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:41.744214 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:41.746801 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:41.746805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:41.746807 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:41.746835 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:41.748545 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:41.749120 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:41.749125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:41.749128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:41.749240 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:41.749708 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:41.749714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:41.749717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:41.750043 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:41.750189 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:41.750432 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 00:53:41.750549 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260806 00:53:41.751900 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 28106
I20260806 00:53:41.765157 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 28287
I20260806 00:53:41.786438 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 28472
I20260806 00:53:41.802397 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 27989
2026-08-06T00:53:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2745 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:41 dist-test-slave-sb2z krb5kdc[28661](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:41 dist-test-slave-sb2z krb5kdc[28661](info): set up 2 sockets
Aug 06 00:53:41 dist-test-slave-sb2z krb5kdc[28661](info): commencing operation
krb5kdc: starting...
W20260806 00:53:43.040323 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:53:43 dist-test-slave-sb2z krb5kdc[28661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:53:43.175917 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:45623
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:45623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:43.296574 28679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:43.296867 28679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:43.296909 28679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:43.301096 28679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:43.301173 28679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:43.301194 28679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:43.301211 28679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:43.301227 28679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:43.307142 28679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:45623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:45623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:43.308395 28679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:43.309495 28679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:43.318737 28685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:43.318749 28688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:43.318791 28684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:43.319154 28679 server_base.cc:1034] running on GCE node
I20260806 00:53:43.320196 28679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:43.322000 28679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:43.323146 28679 hybrid_clock.cc:648] HybridClock initialized: now 1785977623323120 us; error 48 us; skew 500 ppm
Aug 06 00:53:43 dist-test-slave-sb2z krb5kdc[28661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977623, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:43.327219 28679 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:53:43.328377 28679 webserver.cc:460] Webserver started at http://127.27.77.254:44345/ using document root <none> and password file <none>
I20260806 00:53:43.329023 28679 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:43.329113 28679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:43.329337 28679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:43.331514 28679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "77a7837adc76437e9a18f68f1a9ea18e"
format_stamp: "Formatted at 2026-08-06 00:53:43 on dist-test-slave-sb2z"
server_key: "9418d285398cdb62529753d7b6ebf968"
server_key_iv: "daf93c500dc0d845c27549a385062534"
server_key_version: "encryptionkey@0"
I20260806 00:53:43.332197 28679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "77a7837adc76437e9a18f68f1a9ea18e"
format_stamp: "Formatted at 2026-08-06 00:53:43 on dist-test-slave-sb2z"
server_key: "9418d285398cdb62529753d7b6ebf968"
server_key_iv: "daf93c500dc0d845c27549a385062534"
server_key_version: "encryptionkey@0"
I20260806 00:53:43.336403 28679 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:53:43.339112 28698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:43.340580 28679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:53:43.340739 28679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "77a7837adc76437e9a18f68f1a9ea18e"
format_stamp: "Formatted at 2026-08-06 00:53:43 on dist-test-slave-sb2z"
server_key: "9418d285398cdb62529753d7b6ebf968"
server_key_iv: "daf93c500dc0d845c27549a385062534"
server_key_version: "encryptionkey@0"
I20260806 00:53:43.340869 28679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:43.364393 28679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:43.368083 28679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:43.368314 28679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:43.375876 28679 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:45623
I20260806 00:53:43.375906 28766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:45623 every 8 connection(s)
I20260806 00:53:43.376894 28679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:43.379823 28767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:43.384169 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 28679
I20260806 00:53:43.384299 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:53:43.384572 27959 external_mini_cluster.cc:1428] Setting key be32f8af13a6f14878bd79fd9cc1d342
I20260806 00:53:43.386270 28767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e: Bootstrap starting.
I20260806 00:53:43.388664 28767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:43.389457 28767 log.cc:824] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:43.391409 28767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e: No bootstrap required, opened a new log
Aug 06 00:53:43 dist-test-slave-sb2z krb5kdc[28661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:43.396585 28767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a7837adc76437e9a18f68f1a9ea18e" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45623 } }
I20260806 00:53:43.396901 28767 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:43.396955 28767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77a7837adc76437e9a18f68f1a9ea18e, State: Initialized, Role: FOLLOWER
I20260806 00:53:43.397541 28767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a7837adc76437e9a18f68f1a9ea18e" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45623 } }
I20260806 00:53:43.397727 28767 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:43.397804 28767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:43.397915 28767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:43.399399 28767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a7837adc76437e9a18f68f1a9ea18e" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45623 } }
I20260806 00:53:43.399909 28767 leader_election.cc:302] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77a7837adc76437e9a18f68f1a9ea18e; no voters:
I20260806 00:53:43.400352 28767 leader_election.cc:288] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:43.400527 28773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:43.400789 28773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [term 1 LEADER]: Becoming Leader. State: Replica: 77a7837adc76437e9a18f68f1a9ea18e, State: Running, Role: LEADER
I20260806 00:53:43.400801 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:43.386109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51346 (local address 127.27.77.254:45623)
0806 00:53:43.386622 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:43.386637 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:43.386675 (+ 38us) server_negotiation.cc:407] Connection header received
0806 00:53:43.387358 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:43.387376 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:43.387781 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:43.388137 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:43.389025 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:43.390096 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:43.391018 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:43.391490 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:43.391686 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:43.394741 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:43.394751 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:43.394776 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:43.394820 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:43.396937 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:43.397441 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:43.397447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:43.397452 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:43.397608 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:43.398025 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:43.398030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:43.398032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:43.398583 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:43.398778 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:43.399155 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 00:53:43.399638 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":148,"threads_started":1}
I20260806 00:53:43.401160 28773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a7837adc76437e9a18f68f1a9ea18e" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45623 } }
I20260806 00:53:43.401746 28767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:43.402906 28774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77a7837adc76437e9a18f68f1a9ea18e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a7837adc76437e9a18f68f1a9ea18e" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45623 } } }
I20260806 00:53:43.403077 28774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:43.403493 28775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77a7837adc76437e9a18f68f1a9ea18e. Latest consensus state: current_term: 1 leader_uuid: "77a7837adc76437e9a18f68f1a9ea18e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a7837adc76437e9a18f68f1a9ea18e" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45623 } } }
I20260806 00:53:43.403623 28775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e [sys.catalog]: This master's current role is: LEADER
W20260806 00:53:43.407040 28793 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:53:43.407109 28793 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:53:43.407222 28786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:43.408066 28786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:43.413300 28786 catalog_manager.cc:1347] Generated new cluster ID: 8be0c1fdbef74339afe6e15ce613092e
I20260806 00:53:43.413374 28786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:43.422271 28786 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:43.423278 28786 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:43.432066 28786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77a7837adc76437e9a18f68f1a9ea18e: Generated new TSK 0
I20260806 00:53:43.432843 28786 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:53:43.474634 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:45623
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:43.594738 28803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:43.595082 28803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:43.595181 28803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:43.599486 28803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:43.599614 28803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:43.599712 28803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:53:43.605731 28803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:45623
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:43.606992 28803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:43.608233 28803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:43.619802 28809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:43.619820 28810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:43.619827 28814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:43.620759 28803 server_base.cc:1034] running on GCE node
I20260806 00:53:43.621208 28803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:43.621927 28803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:43.623106 28803 hybrid_clock.cc:648] HybridClock initialized: now 1785977623623087 us; error 32 us; skew 500 ppm
Aug 06 00:53:43 dist-test-slave-sb2z krb5kdc[28661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977623, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:43.626185 28803 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:53:43.627254 28803 webserver.cc:460] Webserver started at http://127.27.77.193:44951/ using document root <none> and password file <none>
I20260806 00:53:43.627969 28803 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:43.628078 28803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:43.628343 28803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:43.630638 28803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "cbd00e09c3dc4c45b7d572e21dc073bc"
format_stamp: "Formatted at 2026-08-06 00:53:43 on dist-test-slave-sb2z"
server_key: "4d749e299e816c3838c0736c31701f87"
server_key_iv: "9ee211a2935d18652bcfd8c4f07c3e98"
server_key_version: "encryptionkey@0"
I20260806 00:53:43.631276 28803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "cbd00e09c3dc4c45b7d572e21dc073bc"
format_stamp: "Formatted at 2026-08-06 00:53:43 on dist-test-slave-sb2z"
server_key: "4d749e299e816c3838c0736c31701f87"
server_key_iv: "9ee211a2935d18652bcfd8c4f07c3e98"
server_key_version: "encryptionkey@0"
I20260806 00:53:43.635484 28803 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 00:53:43.638190 28824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:43.639571 28803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:53:43.639717 28803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "cbd00e09c3dc4c45b7d572e21dc073bc"
format_stamp: "Formatted at 2026-08-06 00:53:43 on dist-test-slave-sb2z"
server_key: "4d749e299e816c3838c0736c31701f87"
server_key_iv: "9ee211a2935d18652bcfd8c4f07c3e98"
server_key_version: "encryptionkey@0"
I20260806 00:53:43.639833 28803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:43.659261 28803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:43.662664 28803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:43.662879 28803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:43.663513 28803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:53:43.664430 28803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:53:43.664525 28803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:43.664600 28803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:53:43.664644 28803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:43.674041 28803 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:38005
I20260806 00:53:43.674093 28978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:38005 every 8 connection(s)
I20260806 00:53:43.675163 28803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:53:43.682670 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 28803
I20260806 00:53:43.682813 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:53:43.683099 27959 external_mini_cluster.cc:1428] Setting key 675eb403b4ab461212ea59461b5a35ad
Aug 06 00:53:43 dist-test-slave-sb2z krb5kdc[28661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977623, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:43.689571 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:43.677236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:60557 (local address 127.27.77.254:45623)
0806 00:53:43.677400 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:43.677404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:43.678260 (+ 856us) server_negotiation.cc:407] Connection header received
0806 00:53:43.679203 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:43.679206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:43.679267 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:43.679385 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:43.680639 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:43.681200 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:43.682385 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:43.682738 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:43.682862 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:43.685614 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:43.685621 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:43.685627 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:43.685662 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:43.687242 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:43.687887 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:43.687891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:43.687893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:43.687980 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:43.688411 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:43.688414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:43.688416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:43.688611 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:43.688725 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:43.689250 (+ 525us) server_negotiation.cc:299] Negotiation successful
0806 00:53:43.689391 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:53:43.690433 28982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:43.677605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:45623 (local address 127.27.77.193:60557)
0806 00:53:43.678095 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:43.678140 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:43.678989 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:43.679623 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:43.679630 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:43.679999 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:43.680475 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:43.680488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:43.681349 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:43.681353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:43.682247 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:43.682257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:43.682945 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:43.682955 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:43.683115 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:43.683887 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:43.683905 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:43.685440 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:43.687379 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:43.687385 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:43.687403 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:43.687770 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:43.688137 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:43.688140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:43.688142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:43.688301 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:43.688725 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:43.688734 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:43.688996 (+ 262us) client_negotiation.cc:769] Sending connection context
0806 00:53:43.689195 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 00:53:43.689546 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":110,"threads_started":1}
I20260806 00:53:43.691999 28979 heartbeater.cc:347] Connected to a master server at 127.27.77.254:45623
I20260806 00:53:43.692277 28979 heartbeater.cc:464] Registering TS with master...
I20260806 00:53:43.692845 28979 heartbeater.cc:511] Master 127.27.77.254:45623 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
I20260806 00:53:43.694495 28718 ts_manager.cc:194] Registered new tserver with Master: cbd00e09c3dc4c45b7d572e21dc073bc (127.27.77.193:38005)
I20260806 00:53:43.695772 28718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:60557
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:53:43.714751 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:45623
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:43.832785 28986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:43.833081 28986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:43.833129 28986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:43.837231 28986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:43.837302 28986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:43.837364 28986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:53:43.843072 28986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:45623
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:43.844350 28986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:43.845479 28986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:43.855916 28992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:43.856284 28993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:43.856415 28986 server_base.cc:1034] running on GCE node
W20260806 00:53:43.856453 28995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:43.856902 28986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:43.857689 28986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:43.858896 28986 hybrid_clock.cc:648] HybridClock initialized: now 1785977623858873 us; error 36 us; skew 500 ppm
Aug 06 00:53:43 dist-test-slave-sb2z krb5kdc[28661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977623, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:43.861941 28986 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:53:43.862912 28986 webserver.cc:460] Webserver started at http://127.27.77.194:40465/ using document root <none> and password file <none>
I20260806 00:53:43.863513 28986 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:43.863677 28986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:43.863902 28986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:43.866119 28986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "aea8783f98b24ba1a058e6d2e1fec3d1"
format_stamp: "Formatted at 2026-08-06 00:53:43 on dist-test-slave-sb2z"
server_key: "7f894a7e798363968cd99b13fb41556e"
server_key_iv: "93ff220b4f8c5c3b43512053ee403f5c"
server_key_version: "encryptionkey@0"
I20260806 00:53:43.866727 28986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "aea8783f98b24ba1a058e6d2e1fec3d1"
format_stamp: "Formatted at 2026-08-06 00:53:43 on dist-test-slave-sb2z"
server_key: "7f894a7e798363968cd99b13fb41556e"
server_key_iv: "93ff220b4f8c5c3b43512053ee403f5c"
server_key_version: "encryptionkey@0"
I20260806 00:53:43.870738 28986 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 00:53:43.873370 29004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:43.874792 28986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:53:43.875041 28986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "aea8783f98b24ba1a058e6d2e1fec3d1"
format_stamp: "Formatted at 2026-08-06 00:53:43 on dist-test-slave-sb2z"
server_key: "7f894a7e798363968cd99b13fb41556e"
server_key_iv: "93ff220b4f8c5c3b43512053ee403f5c"
server_key_version: "encryptionkey@0"
I20260806 00:53:43.875202 28986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:43.890125 28986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:43.893445 28986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:43.893666 28986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:43.894279 28986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:53:43.895128 28986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:53:43.895202 28986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:43.895283 28986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:53:43.895327 28986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:43.905516 28986 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:35801
I20260806 00:53:43.905543 29159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:35801 every 8 connection(s)
I20260806 00:53:43.906469 28986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:53:43.912870 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 28986
I20260806 00:53:43.912963 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:53:43.913264 27959 external_mini_cluster.cc:1428] Setting key 55a3605453a949bca6f3b139d16b7f44
Aug 06 00:53:43 dist-test-slave-sb2z krb5kdc[28661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977623, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:43.920445 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:43.908337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:35601 (local address 127.27.77.254:45623)
0806 00:53:43.908508 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:43.908513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:43.909628 (+ 1115us) server_negotiation.cc:407] Connection header received
0806 00:53:43.910496 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:43.910499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:43.910547 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:43.910626 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:43.911899 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:43.912421 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:43.913640 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:43.913875 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:43.913997 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:43.916808 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:43.916811 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:43.916813 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:43.916837 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:43.918250 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:43.918782 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:43.918789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:43.918791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:43.918877 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:43.919249 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:43.919253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:43.919254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:43.919455 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:43.919598 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:43.920145 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 00:53:43.920280 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:53:43.921334 29162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:43.908772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:45623 (local address 127.27.77.194:35601)
0806 00:53:43.909474 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:43.909516 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:43.910290 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:43.910791 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:43.910797 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:43.911191 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:43.911745 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:43.911760 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:43.912534 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:43.912537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:43.913516 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:43.913527 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:43.914041 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:43.914045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:43.914222 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:43.914844 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:43.914861 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:43.916616 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:43.918374 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:43.918380 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:43.918396 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:43.918671 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:43.918975 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:43.918979 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:43.918981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:43.919137 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:43.919599 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:43.919607 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:43.919876 (+ 269us) client_negotiation.cc:769] Sending connection context
0806 00:53:43.920098 (+ 222us) client_negotiation.cc:240] Negotiation successful
0806 00:53:43.920442 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":106,"threads_started":1}
I20260806 00:53:43.922608 29160 heartbeater.cc:347] Connected to a master server at 127.27.77.254:45623
I20260806 00:53:43.922914 29160 heartbeater.cc:464] Registering TS with master...
I20260806 00:53:43.923687 29160 heartbeater.cc:511] Master 127.27.77.254:45623 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
I20260806 00:53:43.925009 28718 ts_manager.cc:194] Registered new tserver with Master: aea8783f98b24ba1a058e6d2e1fec3d1 (127.27.77.194:35801)
I20260806 00:53:43.925685 28718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:35601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:53:43.946875 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:45623
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:44.065568 29166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:44.065905 29166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:44.066021 29166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:44.070878 29166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:44.070957 29166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:44.071074 29166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:53:44.077389 29166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:45623
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:44.078675 29166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:44.079948 29166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:44.090847 29173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:44.090898 29174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:44.090994 29166 server_base.cc:1034] running on GCE node
W20260806 00:53:44.090886 29176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:44.091611 29166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:44.092304 29166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:44.093509 29166 hybrid_clock.cc:648] HybridClock initialized: now 1785977624093492 us; error 36 us; skew 500 ppm
Aug 06 00:53:44 dist-test-slave-sb2z krb5kdc[28661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977624, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:44.096825 29166 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:53:44.097816 29166 webserver.cc:460] Webserver started at http://127.27.77.195:38591/ using document root <none> and password file <none>
I20260806 00:53:44.098461 29166 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:44.098565 29166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:44.098824 29166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:44.101189 29166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "c7f5a7bad671499cab9e701da8db63ed"
format_stamp: "Formatted at 2026-08-06 00:53:44 on dist-test-slave-sb2z"
server_key: "a8fc991986941ef5cb6c6195dbb675fa"
server_key_iv: "bf2465a4586ce43cfe539d6fd31e6212"
server_key_version: "encryptionkey@0"
I20260806 00:53:44.101871 29166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "c7f5a7bad671499cab9e701da8db63ed"
format_stamp: "Formatted at 2026-08-06 00:53:44 on dist-test-slave-sb2z"
server_key: "a8fc991986941ef5cb6c6195dbb675fa"
server_key_iv: "bf2465a4586ce43cfe539d6fd31e6212"
server_key_version: "encryptionkey@0"
I20260806 00:53:44.106665 29166 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 00:53:44.109524 29184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:44.111317 29166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:53:44.111469 29166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "c7f5a7bad671499cab9e701da8db63ed"
format_stamp: "Formatted at 2026-08-06 00:53:44 on dist-test-slave-sb2z"
server_key: "a8fc991986941ef5cb6c6195dbb675fa"
server_key_iv: "bf2465a4586ce43cfe539d6fd31e6212"
server_key_version: "encryptionkey@0"
I20260806 00:53:44.111639 29166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:44.131075 29166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:44.134662 29166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:44.134899 29166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:44.135612 29166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:53:44.136520 29166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:53:44.136572 29166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:44.136613 29166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:53:44.136652 29166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:44.146848 29166 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:41401
I20260806 00:53:44.146874 29351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:41401 every 8 connection(s)
I20260806 00:53:44.147900 29166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:53:44.155313 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 29166
I20260806 00:53:44.155476 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:53:44.155866 27959 external_mini_cluster.cc:1428] Setting key 82d6b333acbe34dfe1464bbff19c5fd0
Aug 06 00:53:44 dist-test-slave-sb2z krb5kdc[28661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977624, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:44.162582 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.149942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:40187 (local address 127.27.77.254:45623)
0806 00:53:44.150135 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:44.150140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:44.151288 (+ 1148us) server_negotiation.cc:407] Connection header received
0806 00:53:44.152369 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:44.152372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:44.152421 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:44.152498 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:44.153771 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.154326 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.155681 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.156024 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.156150 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.159077 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:44.159080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:44.159082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:44.159106 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:44.160530 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:44.161018 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:44.161021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:44.161023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:44.161104 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:44.161480 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:44.161484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:44.161485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:44.161694 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:44.161795 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:44.162329 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 00:53:44.162445 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 00:53:44.163636 29357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.150333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:45623 (local address 127.27.77.195:40187)
0806 00:53:44.151118 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:44.151167 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:44.152165 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:44.152680 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:44.152687 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:44.153109 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:44.153634 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:44.153649 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.154444 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.154447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:44.155426 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:44.155466 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.156260 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.156265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:44.156484 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.157196 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:44.157220 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:44.158904 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:44.160638 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:44.160643 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:44.160655 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:44.160914 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:44.161191 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:44.161193 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:44.161195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:44.161344 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:44.161801 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:44.161806 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:44.162087 (+ 281us) client_negotiation.cc:769] Sending connection context
0806 00:53:44.162257 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 00:53:44.162532 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":132,"threads_started":1}
I20260806 00:53:44.164884 29354 heartbeater.cc:347] Connected to a master server at 127.27.77.254:45623
I20260806 00:53:44.165166 29354 heartbeater.cc:464] Registering TS with master...
I20260806 00:53:44.165745 29354 heartbeater.cc:511] Master 127.27.77.254:45623 requested a full tablet report, sending...
I20260806 00:53:44.166891 28718 ts_manager.cc:194] Registered new tserver with Master: c7f5a7bad671499cab9e701da8db63ed (127.27.77.195:41401)
I20260806 00:53:44.167593 28718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:40187
I20260806 00:53:44.169998 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:53:44.182040 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.173789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51362 (local address 127.27.77.254:45623)
0806 00:53:44.173924 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:44.173927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:44.174092 (+ 165us) server_negotiation.cc:407] Connection header received
0806 00:53:44.174204 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:44.174206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:44.174247 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:44.174324 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:44.175091 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.175672 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.176627 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.176861 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.176960 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.178394 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:44.178397 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:44.178399 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:44.178425 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:44.180181 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:44.180695 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:44.180698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:44.180699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:44.180795 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:44.181122 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:44.181125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:44.181127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:44.181389 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:44.181513 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:44.181775 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 00:53:44.181902 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:53:44.187456 28718 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:53:44.190565 28718 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:53:44.206180 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.199726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:38005 (local address 127.0.0.1:49988)
0806 00:53:44.200516 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:44.200529 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:44.200623 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:44.201436 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:44.201438 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:44.201453 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:44.201534 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:44.201542 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.203024 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.203027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:44.204587 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:44.204595 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.205627 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.205630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:44.205722 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.205754 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 00:53:44.205884 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 00:53:44.206017 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":88,"thread_start_us":125,"threads_started":1}
I20260806 00:53:44.206225 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.200369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49988 (local address 127.27.77.193:38005)
0806 00:53:44.200786 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:44.200794 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:44.200820 (+ 26us) server_negotiation.cc:407] Connection header received
0806 00:53:44.200900 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:44.200907 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:44.201135 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:44.201289 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:44.201670 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.202908 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.204695 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.205483 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.205590 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.205741 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:44.205928 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 00:53:44.206042 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":90,"threads_started":1}
I20260806 00:53:44.207152 29370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.199509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:41401 (local address 127.0.0.1:54144)
0806 00:53:44.200045 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:44.200096 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:44.200252 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:44.201309 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:44.201315 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:44.201337 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:44.201468 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:44.201475 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.202863 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.202869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:44.204583 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:44.204595 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.206596 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.206599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:44.206790 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.206829 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 00:53:44.206898 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 00:53:44.206988 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":190,"thread_start_us":75,"threads_started":1}
I20260806 00:53:44.207199 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.199736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54144 (local address 127.27.77.195:41401)
0806 00:53:44.200861 (+ 1125us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:44.200872 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:44.200890 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:53:44.200973 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:44.200977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:44.201131 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:44.201637 (+ 506us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:44.201847 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.202714 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.205814 (+ 3100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.206462 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.206738 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.206898 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:44.206950 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 00:53:44.207046 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1029,"thread_start_us":237,"threads_started":1}
I20260806 00:53:44.207602 29376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.200584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55490 (local address 127.27.77.194:35801)
0806 00:53:44.201183 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:44.201197 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:44.201220 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:53:44.201293 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:44.201299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:44.201535 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:44.201702 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:44.201949 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.203146 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.204629 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.205473 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.205591 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.205741 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:44.207323 (+ 1582us) server_negotiation.cc:299] Negotiation successful
0806 00:53:44.207427 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":123,"threads_started":1}
I20260806 00:53:44.207046 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.200242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:35801 (local address 127.0.0.1:55490)
0806 00:53:44.200604 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:44.200618 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:44.200719 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:44.201678 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:44.201681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:44.201699 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:44.201812 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:44.201821 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.203278 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.203283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:44.204497 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:44.204508 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.206629 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.206632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:44.206742 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.206784 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 00:53:44.206855 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 00:53:44.206919 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":8,"thread_start_us":111,"threads_started":1}
I20260806 00:53:44.209484 28892 tablet_service.cc:1467] Processing CreateTablet for tablet 519b7d5ea6af412398f50fb99681f96e (DEFAULT_TABLE table=test-table [id=fbbe542432d24802b2160e0f5b3114c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:44.210565 28892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 519b7d5ea6af412398f50fb99681f96e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:44.210201 29074 tablet_service.cc:1467] Processing CreateTablet for tablet 519b7d5ea6af412398f50fb99681f96e (DEFAULT_TABLE table=test-table [id=fbbe542432d24802b2160e0f5b3114c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:44.211324 29074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 519b7d5ea6af412398f50fb99681f96e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:44.212702 29259 tablet_service.cc:1467] Processing CreateTablet for tablet 519b7d5ea6af412398f50fb99681f96e (DEFAULT_TABLE table=test-table [id=fbbe542432d24802b2160e0f5b3114c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:44.213802 29259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 519b7d5ea6af412398f50fb99681f96e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:44.217695 29377 tablet_bootstrap.cc:492] T 519b7d5ea6af412398f50fb99681f96e P cbd00e09c3dc4c45b7d572e21dc073bc: Bootstrap starting.
I20260806 00:53:44.220042 29378 tablet_bootstrap.cc:492] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1: Bootstrap starting.
I20260806 00:53:44.220526 29377 tablet_bootstrap.cc:654] T 519b7d5ea6af412398f50fb99681f96e P cbd00e09c3dc4c45b7d572e21dc073bc: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:44.221148 29379 tablet_bootstrap.cc:492] T 519b7d5ea6af412398f50fb99681f96e P c7f5a7bad671499cab9e701da8db63ed: Bootstrap starting.
I20260806 00:53:44.221669 29377 log.cc:824] T 519b7d5ea6af412398f50fb99681f96e P cbd00e09c3dc4c45b7d572e21dc073bc: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:44.222811 29378 tablet_bootstrap.cc:654] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:44.223184 29379 tablet_bootstrap.cc:654] T 519b7d5ea6af412398f50fb99681f96e P c7f5a7bad671499cab9e701da8db63ed: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:44.223744 29378 log.cc:824] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:44.224164 29377 tablet_bootstrap.cc:492] T 519b7d5ea6af412398f50fb99681f96e P cbd00e09c3dc4c45b7d572e21dc073bc: No bootstrap required, opened a new log
I20260806 00:53:44.224411 29377 ts_tablet_manager.cc:1397] T 519b7d5ea6af412398f50fb99681f96e P cbd00e09c3dc4c45b7d572e21dc073bc: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:53:44.224967 29379 log.cc:824] T 519b7d5ea6af412398f50fb99681f96e P c7f5a7bad671499cab9e701da8db63ed: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:44.225819 29378 tablet_bootstrap.cc:492] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1: No bootstrap required, opened a new log
I20260806 00:53:44.226020 29378 ts_tablet_manager.cc:1397] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:53:44.226794 29379 tablet_bootstrap.cc:492] T 519b7d5ea6af412398f50fb99681f96e P c7f5a7bad671499cab9e701da8db63ed: No bootstrap required, opened a new log
I20260806 00:53:44.227008 29379 ts_tablet_manager.cc:1397] T 519b7d5ea6af412398f50fb99681f96e P c7f5a7bad671499cab9e701da8db63ed: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 00:53:44.229434 29377 raft_consensus.cc:348] T 519b7d5ea6af412398f50fb99681f96e P cbd00e09c3dc4c45b7d572e21dc073bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f5a7bad671499cab9e701da8db63ed" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41401 } } peers { permanent_uuid: "cbd00e09c3dc4c45b7d572e21dc073bc" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38005 } } peers { permanent_uuid: "aea8783f98b24ba1a058e6d2e1fec3d1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35801 } }
I20260806 00:53:44.229741 29377 raft_consensus.cc:374] T 519b7d5ea6af412398f50fb99681f96e P cbd00e09c3dc4c45b7d572e21dc073bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:44.229807 29377 raft_consensus.cc:729] T 519b7d5ea6af412398f50fb99681f96e P cbd00e09c3dc4c45b7d572e21dc073bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbd00e09c3dc4c45b7d572e21dc073bc, State: Initialized, Role: FOLLOWER
I20260806 00:53:44.230134 29378 raft_consensus.cc:348] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f5a7bad671499cab9e701da8db63ed" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41401 } } peers { permanent_uuid: "cbd00e09c3dc4c45b7d572e21dc073bc" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38005 } } peers { permanent_uuid: "aea8783f98b24ba1a058e6d2e1fec3d1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35801 } }
I20260806 00:53:44.230283 29377 consensus_queue.cc:260] T 519b7d5ea6af412398f50fb99681f96e P cbd00e09c3dc4c45b7d572e21dc073bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f5a7bad671499cab9e701da8db63ed" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41401 } } peers { permanent_uuid: "cbd00e09c3dc4c45b7d572e21dc073bc" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38005 } } peers { permanent_uuid: "aea8783f98b24ba1a058e6d2e1fec3d1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35801 } }
I20260806 00:53:44.230196 29379 raft_consensus.cc:348] T 519b7d5ea6af412398f50fb99681f96e P c7f5a7bad671499cab9e701da8db63ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f5a7bad671499cab9e701da8db63ed" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41401 } } peers { permanent_uuid: "cbd00e09c3dc4c45b7d572e21dc073bc" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38005 } } peers { permanent_uuid: "aea8783f98b24ba1a058e6d2e1fec3d1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35801 } }
I20260806 00:53:44.230403 29378 raft_consensus.cc:374] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:44.230427 29379 raft_consensus.cc:374] T 519b7d5ea6af412398f50fb99681f96e P c7f5a7bad671499cab9e701da8db63ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:44.230458 29378 raft_consensus.cc:729] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aea8783f98b24ba1a058e6d2e1fec3d1, State: Initialized, Role: FOLLOWER
I20260806 00:53:44.230477 29379 raft_consensus.cc:729] T 519b7d5ea6af412398f50fb99681f96e P c7f5a7bad671499cab9e701da8db63ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f5a7bad671499cab9e701da8db63ed, State: Initialized, Role: FOLLOWER
I20260806 00:53:44.230881 29378 consensus_queue.cc:260] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f5a7bad671499cab9e701da8db63ed" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41401 } } peers { permanent_uuid: "cbd00e09c3dc4c45b7d572e21dc073bc" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38005 } } peers { permanent_uuid: "aea8783f98b24ba1a058e6d2e1fec3d1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35801 } }
I20260806 00:53:44.230911 29379 consensus_queue.cc:260] T 519b7d5ea6af412398f50fb99681f96e P c7f5a7bad671499cab9e701da8db63ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f5a7bad671499cab9e701da8db63ed" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41401 } } peers { permanent_uuid: "cbd00e09c3dc4c45b7d572e21dc073bc" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38005 } } peers { permanent_uuid: "aea8783f98b24ba1a058e6d2e1fec3d1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35801 } }
I20260806 00:53:44.231372 29377 ts_tablet_manager.cc:1428] T 519b7d5ea6af412398f50fb99681f96e P cbd00e09c3dc4c45b7d572e21dc073bc: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 00:53:44.231774 28979 heartbeater.cc:503] Master 127.27.77.254:45623 was elected leader, sending a full tablet report...
I20260806 00:53:44.231828 29378 ts_tablet_manager.cc:1428] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 00:53:44.231880 29379 ts_tablet_manager.cc:1428] T 519b7d5ea6af412398f50fb99681f96e P c7f5a7bad671499cab9e701da8db63ed: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260806 00:53:44.232143 29160 heartbeater.cc:503] Master 127.27.77.254:45623 was elected leader, sending a full tablet report...
I20260806 00:53:44.232223 29354 heartbeater.cc:503] Master 127.27.77.254:45623 was elected leader, sending a full tablet report...
W20260806 00:53:44.399041 29356 tablet.cc:2369] T 519b7d5ea6af412398f50fb99681f96e P c7f5a7bad671499cab9e701da8db63ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:53:44.407761 29161 tablet.cc:2369] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:53:44.426934 28980 tablet.cc:2369] T 519b7d5ea6af412398f50fb99681f96e P cbd00e09c3dc4c45b7d572e21dc073bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:53:44.564041 29386 raft_consensus.cc:482] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:53:44.564338 29386 raft_consensus.cc:504] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f5a7bad671499cab9e701da8db63ed" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41401 } } peers { permanent_uuid: "cbd00e09c3dc4c45b7d572e21dc073bc" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38005 } } peers { permanent_uuid: "aea8783f98b24ba1a058e6d2e1fec3d1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35801 } }
I20260806 00:53:44.566318 29386 leader_election.cc:288] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7f5a7bad671499cab9e701da8db63ed (127.27.77.195:41401), cbd00e09c3dc4c45b7d572e21dc073bc (127.27.77.193:38005)
I20260806 00:53:44.569818 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.566187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:36295 (local address 127.27.77.195:41401)
0806 00:53:44.566340 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:44.566347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:44.566486 (+ 139us) server_negotiation.cc:407] Connection header received
0806 00:53:44.566659 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:44.566662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:44.566714 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:44.566797 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:44.567100 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.567699 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.568774 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.569241 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.569373 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.569430 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:44.569615 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 00:53:44.569686 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:53:44.569844 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.565970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:41401 (local address 127.27.77.194:36295)
0806 00:53:44.566357 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:44.566373 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:44.566534 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:44.566821 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:44.566824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:44.566841 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:44.566985 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:44.566991 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.567808 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.567813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:44.568643 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:44.568652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.569373 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.569377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:44.569481 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.569510 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 00:53:44.569599 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 00:53:44.569682 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":120,"threads_started":1}
I20260806 00:53:44.569967 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.566176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:50809 (local address 127.27.77.193:38005)
0806 00:53:44.566357 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:44.566363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:44.566684 (+ 321us) server_negotiation.cc:407] Connection header received
0806 00:53:44.566851 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:44.566854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:44.566906 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:44.566993 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:44.567312 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.567831 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.568921 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.569364 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.569500 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.569551 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:44.569749 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 00:53:44.569823 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 00:53:44.570201 29388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.566025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:38005 (local address 127.27.77.194:50809)
0806 00:53:44.566585 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:44.566597 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:44.566728 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:44.567007 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:44.567010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:44.567028 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:44.567182 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:44.567189 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.567948 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.567953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:44.568787 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:44.568797 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.569492 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.569495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:44.569594 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.569616 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:53:44.569718 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 00:53:44.569794 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":89,"threads_started":1}
I20260806 00:53:44.570539 29287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "519b7d5ea6af412398f50fb99681f96e" candidate_uuid: "aea8783f98b24ba1a058e6d2e1fec3d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f5a7bad671499cab9e701da8db63ed" is_pre_election: true
I20260806 00:53:44.570919 29287 raft_consensus.cc:2454] T 519b7d5ea6af412398f50fb99681f96e P c7f5a7bad671499cab9e701da8db63ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aea8783f98b24ba1a058e6d2e1fec3d1 in term 0.
I20260806 00:53:44.570901 28914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "519b7d5ea6af412398f50fb99681f96e" candidate_uuid: "aea8783f98b24ba1a058e6d2e1fec3d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd00e09c3dc4c45b7d572e21dc073bc" is_pre_election: true
I20260806 00:53:44.571210 28914 raft_consensus.cc:2454] T 519b7d5ea6af412398f50fb99681f96e P cbd00e09c3dc4c45b7d572e21dc073bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aea8783f98b24ba1a058e6d2e1fec3d1 in term 0.
I20260806 00:53:44.571420 29007 leader_election.cc:302] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aea8783f98b24ba1a058e6d2e1fec3d1, c7f5a7bad671499cab9e701da8db63ed; no voters:
I20260806 00:53:44.571754 29386 raft_consensus.cc:2781] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:53:44.571864 29386 raft_consensus.cc:482] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:53:44.571913 29386 raft_consensus.cc:3037] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:44.573007 29386 raft_consensus.cc:504] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f5a7bad671499cab9e701da8db63ed" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41401 } } peers { permanent_uuid: "cbd00e09c3dc4c45b7d572e21dc073bc" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38005 } } peers { permanent_uuid: "aea8783f98b24ba1a058e6d2e1fec3d1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35801 } }
I20260806 00:53:44.573410 29386 leader_election.cc:288] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [CANDIDATE]: Term 1 election: Requested vote from peers c7f5a7bad671499cab9e701da8db63ed (127.27.77.195:41401), cbd00e09c3dc4c45b7d572e21dc073bc (127.27.77.193:38005)
I20260806 00:53:44.573886 29287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "519b7d5ea6af412398f50fb99681f96e" candidate_uuid: "aea8783f98b24ba1a058e6d2e1fec3d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f5a7bad671499cab9e701da8db63ed"
I20260806 00:53:44.573923 28914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "519b7d5ea6af412398f50fb99681f96e" candidate_uuid: "aea8783f98b24ba1a058e6d2e1fec3d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd00e09c3dc4c45b7d572e21dc073bc"
I20260806 00:53:44.574034 29287 raft_consensus.cc:3037] T 519b7d5ea6af412398f50fb99681f96e P c7f5a7bad671499cab9e701da8db63ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:44.574052 28914 raft_consensus.cc:3037] T 519b7d5ea6af412398f50fb99681f96e P cbd00e09c3dc4c45b7d572e21dc073bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:44.575146 29287 raft_consensus.cc:2454] T 519b7d5ea6af412398f50fb99681f96e P c7f5a7bad671499cab9e701da8db63ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aea8783f98b24ba1a058e6d2e1fec3d1 in term 1.
I20260806 00:53:44.575220 28914 raft_consensus.cc:2454] T 519b7d5ea6af412398f50fb99681f96e P cbd00e09c3dc4c45b7d572e21dc073bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aea8783f98b24ba1a058e6d2e1fec3d1 in term 1.
I20260806 00:53:44.575490 29007 leader_election.cc:302] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aea8783f98b24ba1a058e6d2e1fec3d1, c7f5a7bad671499cab9e701da8db63ed; no voters:
I20260806 00:53:44.575692 29386 raft_consensus.cc:2781] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:44.575954 29386 raft_consensus.cc:686] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [term 1 LEADER]: Becoming Leader. State: Replica: aea8783f98b24ba1a058e6d2e1fec3d1, State: Running, Role: LEADER
I20260806 00:53:44.576287 29386 consensus_queue.cc:237] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f5a7bad671499cab9e701da8db63ed" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41401 } } peers { permanent_uuid: "cbd00e09c3dc4c45b7d572e21dc073bc" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38005 } } peers { permanent_uuid: "aea8783f98b24ba1a058e6d2e1fec3d1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35801 } }
I20260806 00:53:44.579931 28717 catalog_manager.cc:5495] T 519b7d5ea6af412398f50fb99681f96e P aea8783f98b24ba1a058e6d2e1fec3d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to aea8783f98b24ba1a058e6d2e1fec3d1 (127.27.77.194). New cstate: current_term: 1 leader_uuid: "aea8783f98b24ba1a058e6d2e1fec3d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f5a7bad671499cab9e701da8db63ed" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbd00e09c3dc4c45b7d572e21dc073bc" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aea8783f98b24ba1a058e6d2e1fec3d1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35801 } health_report { overall_health: HEALTHY } } }
Aug 06 00:53:44 dist-test-slave-sb2z krb5kdc[28661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.193@KRBTEST.COM
I20260806 00:53:44.664950 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.655503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50000 (local address 127.27.77.193:38005)
0806 00:53:44.655730 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:44.655735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:44.655885 (+ 150us) server_negotiation.cc:407] Connection header received
0806 00:53:44.656057 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:44.656062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:44.656112 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:44.656204 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:44.656925 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.657458 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.658345 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.658565 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.658690 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.661336 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:44.661351 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:44.661368 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:44.661394 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:44.662966 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:44.663428 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:44.663434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:44.663435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:44.663583 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:44.663873 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:44.663877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:44.663879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:44.664212 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:44.664440 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:44.664647 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 00:53:44.664782 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
Aug 06 00:53:44 dist-test-slave-sb2z krb5kdc[28661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977624, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 00:53:44 dist-test-slave-sb2z krb5kdc[28661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977624, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.77.193@KRBTEST.COM
I20260806 00:53:44.695227 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.686549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50008 (local address 127.27.77.193:38005)
0806 00:53:44.686744 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:44.686748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:44.686843 (+ 95us) server_negotiation.cc:407] Connection header received
0806 00:53:44.686961 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:44.686963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:44.687004 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:44.687075 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:44.687893 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.688411 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.689314 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.689533 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.689614 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.692162 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:44.692166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:44.692168 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:44.692190 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:44.693445 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:44.693898 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:44.693902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:44.693903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:44.694013 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:44.694286 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:44.694289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:44.694290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:44.694474 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:44.694597 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:44.694870 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 00:53:44.695023 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
Aug 06 00:53:44 dist-test-slave-sb2z krb5kdc[28661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 00:53:44 dist-test-slave-sb2z krb5kdc[28661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.193@KRBTEST.COM
I20260806 00:53:44.725144 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:44.716115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50010 (local address 127.27.77.193:38005)
0806 00:53:44.716291 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:44.716295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:44.716365 (+ 70us) server_negotiation.cc:407] Connection header received
0806 00:53:44.716495 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:44.716497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:44.716544 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:44.716627 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:44.717507 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.718027 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.718886 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:44.719126 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:44.719206 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:44.721796 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:44.721799 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:44.721802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:44.721828 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:44.723361 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:44.723841 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:44.723845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:44.723846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:44.723936 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:44.724190 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:44.724193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:44.724194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:44.724378 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:44.724483 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:44.724740 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 00:53:44.724911 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 00:53:44.726800 28863 tablet_service.cc:2908] Rejecting scan request for tablet 519b7d5ea6af412398f50fb99681f96e: Uninitialized: clean time has not yet been initialized
I20260806 00:53:44.729018 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 28803
I20260806 00:53:44.743043 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 28986
I20260806 00:53:44.755261 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 29166
I20260806 00:53:44.765156 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 28679
2026-08-06T00:53:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2963 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:44 dist-test-slave-sb2z krb5kdc[29409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:44 dist-test-slave-sb2z krb5kdc[29409](info): set up 2 sockets
Aug 06 00:53:44 dist-test-slave-sb2z krb5kdc[29409](info): commencing operation
krb5kdc: starting...
W20260806 00:53:46.010982 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:53:46 dist-test-slave-sb2z krb5kdc[29409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:53:46.148890 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:43793
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:43793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:46.264442 29429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:46.264768 29429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:46.264850 29429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:46.268932 29429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:46.269021 29429 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 00:53:46.269052 29429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:46.269095 29429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:46.269135 29429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:46.269170 29429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:46.274477 29429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:43793
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:43793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:46.275648 29429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:46.276697 29429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:46.285476 29435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:46.285439 29434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:46.285529 29437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:46.285849 29429 server_base.cc:1034] running on GCE node
I20260806 00:53:46.286512 29429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:46.287755 29429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:46.288980 29429 hybrid_clock.cc:648] HybridClock initialized: now 1785977626288958 us; error 33 us; skew 500 ppm
Aug 06 00:53:46 dist-test-slave-sb2z krb5kdc[29409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977626, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:46.297674 29429 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:53:46.298722 29429 webserver.cc:460] Webserver started at http://127.27.77.254:39625/ using document root <none> and password file <none>
I20260806 00:53:46.299319 29429 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:46.299386 29429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:46.299644 29429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:46.301905 29429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "929cbad3c1b648f79f1ea260409f7d08"
format_stamp: "Formatted at 2026-08-06 00:53:46 on dist-test-slave-sb2z"
server_key: "2db7f5f3cf04cb05dcc10fd11b4083c1"
server_key_iv: "edea2a2f2f422eaa329b9611fc94456e"
server_key_version: "encryptionkey@0"
I20260806 00:53:46.302465 29429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "929cbad3c1b648f79f1ea260409f7d08"
format_stamp: "Formatted at 2026-08-06 00:53:46 on dist-test-slave-sb2z"
server_key: "2db7f5f3cf04cb05dcc10fd11b4083c1"
server_key_iv: "edea2a2f2f422eaa329b9611fc94456e"
server_key_version: "encryptionkey@0"
I20260806 00:53:46.306744 29429 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:53:46.309415 29452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:46.310734 29429 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:53:46.310844 29429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "929cbad3c1b648f79f1ea260409f7d08"
format_stamp: "Formatted at 2026-08-06 00:53:46 on dist-test-slave-sb2z"
server_key: "2db7f5f3cf04cb05dcc10fd11b4083c1"
server_key_iv: "edea2a2f2f422eaa329b9611fc94456e"
server_key_version: "encryptionkey@0"
I20260806 00:53:46.310932 29429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:46.342291 29429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:46.345502 29429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:46.345683 29429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:46.354743 29429 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:43793
I20260806 00:53:46.354772 29534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:43793 every 8 connection(s)
I20260806 00:53:46.355821 29429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:46.357089 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 29429
I20260806 00:53:46.357185 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:53:46.357456 27959 external_mini_cluster.cc:1428] Setting key 079ddfd9e52ee12ff6eb25fb316aa9eb
I20260806 00:53:46.358986 29535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:46.366088 29535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08: Bootstrap starting.
Aug 06 00:53:46 dist-test-slave-sb2z krb5kdc[29409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:46.368848 29535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:46.369715 29535 log.cc:824] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:46.372059 29535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08: No bootstrap required, opened a new log
I20260806 00:53:46.373809 29541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:46.358902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.27.77.254:43793)
0806 00:53:46.359363 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:46.359384 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:46.359428 (+ 44us) server_negotiation.cc:407] Connection header received
0806 00:53:46.360302 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:46.360326 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:46.360735 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:46.361121 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:46.361849 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.363364 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.364295 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.365004 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.365175 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:46.368171 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:46.368176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:46.368189 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:46.368219 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:46.370854 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:46.371363 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:46.371368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:46.371372 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:46.371495 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:46.371875 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:46.371877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:46.371878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:46.372334 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:46.372502 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:46.372784 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 00:53:46.373090 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":166,"threads_started":1}
I20260806 00:53:46.375483 29535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929cbad3c1b648f79f1ea260409f7d08" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43793 } }
I20260806 00:53:46.375773 29535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:46.375846 29535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929cbad3c1b648f79f1ea260409f7d08, State: Initialized, Role: FOLLOWER
I20260806 00:53:46.376289 29535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929cbad3c1b648f79f1ea260409f7d08" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43793 } }
I20260806 00:53:46.376449 29535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:46.376539 29535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:46.376662 29535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:46.377538 29535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929cbad3c1b648f79f1ea260409f7d08" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43793 } }
I20260806 00:53:46.377873 29535 leader_election.cc:302] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 929cbad3c1b648f79f1ea260409f7d08; no voters:
I20260806 00:53:46.378175 29535 leader_election.cc:288] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:46.378307 29543 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:46.378554 29543 raft_consensus.cc:686] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [term 1 LEADER]: Becoming Leader. State: Replica: 929cbad3c1b648f79f1ea260409f7d08, State: Running, Role: LEADER
I20260806 00:53:46.378849 29543 consensus_queue.cc:237] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929cbad3c1b648f79f1ea260409f7d08" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43793 } }
I20260806 00:53:46.379474 29535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:46.380678 29545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 929cbad3c1b648f79f1ea260409f7d08. Latest consensus state: current_term: 1 leader_uuid: "929cbad3c1b648f79f1ea260409f7d08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929cbad3c1b648f79f1ea260409f7d08" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43793 } } }
I20260806 00:53:46.380821 29545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:46.381182 29544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "929cbad3c1b648f79f1ea260409f7d08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929cbad3c1b648f79f1ea260409f7d08" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43793 } } }
I20260806 00:53:46.381282 29544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:46.381821 29556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:46.384541 29556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:46.389364 29556 catalog_manager.cc:1347] Generated new cluster ID: 197b890f8e244a748bd63c0c7f11d924
I20260806 00:53:46.389428 29556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:46.401628 29556 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:46.402559 29556 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:46.408851 29556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08: Generated new TSK 0
I20260806 00:53:46.409556 29556 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:53:46.446345 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:43793
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:46.563059 29572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:46.563334 29572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:46.563380 29572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:46.567299 29572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:46.567358 29572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:46.567396 29572 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:53:46.567430 29572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:53:46.572667 29572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.77.254:43793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:46.573683 29572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:46.574674 29572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:46.584968 29581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:46.584955 29583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:46.585021 29579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:46.585175 29572 server_base.cc:1034] running on GCE node
I20260806 00:53:46.585779 29572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:46.586416 29572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:46.587646 29572 hybrid_clock.cc:648] HybridClock initialized: now 1785977626587622 us; error 28 us; skew 500 ppm
Aug 06 00:53:46 dist-test-slave-sb2z krb5kdc[29409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977626, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:46.590533 29572 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:53:46.591382 29572 webserver.cc:460] Webserver started at http://127.27.77.193:41825/ using document root <none> and password file <none>
I20260806 00:53:46.592007 29572 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:46.592111 29572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:46.592346 29572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:46.594420 29572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "ae4f2afdb7644fb3b21179127b0b956c"
format_stamp: "Formatted at 2026-08-06 00:53:46 on dist-test-slave-sb2z"
server_key: "e4306a04111aebf81d2899e8853a9eb6"
server_key_iv: "0f50ef8e9162ab50e67c4293b9d58efb"
server_key_version: "encryptionkey@0"
I20260806 00:53:46.595000 29572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "ae4f2afdb7644fb3b21179127b0b956c"
format_stamp: "Formatted at 2026-08-06 00:53:46 on dist-test-slave-sb2z"
server_key: "e4306a04111aebf81d2899e8853a9eb6"
server_key_iv: "0f50ef8e9162ab50e67c4293b9d58efb"
server_key_version: "encryptionkey@0"
I20260806 00:53:46.598722 29572 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:53:46.601226 29594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:46.602476 29572 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 00:53:46.602626 29572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "ae4f2afdb7644fb3b21179127b0b956c"
format_stamp: "Formatted at 2026-08-06 00:53:46 on dist-test-slave-sb2z"
server_key: "e4306a04111aebf81d2899e8853a9eb6"
server_key_iv: "0f50ef8e9162ab50e67c4293b9d58efb"
server_key_version: "encryptionkey@0"
I20260806 00:53:46.602746 29572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:46.621434 29572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:46.625044 29572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:46.625309 29572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:46.627401 29572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:53:46.627467 29572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:46.627558 29572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:53:46.627592 29572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:53:46 dist-test-slave-sb2z krb5kdc[29409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977626, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:46.639650 29572 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:36265
I20260806 00:53:46.639917 29752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:36265 every 8 connection(s)
I20260806 00:53:46.640738 29572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:53:46.642750 29541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:46.628831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:50955 (local address 127.27.77.254:43793)
0806 00:53:46.628968 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:46.628972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:46.630191 (+ 1219us) server_negotiation.cc:407] Connection header received
0806 00:53:46.631726 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:46.631730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:46.631796 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:46.631912 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:46.633249 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.633907 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.635085 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.635418 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.635579 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:46.638693 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:46.638696 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:46.638698 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:46.638723 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:46.640398 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:46.641036 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:46.641040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:46.641042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:46.641124 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:46.641605 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:46.641611 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:46.641614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:46.641843 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:46.641929 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:46.642473 (+ 544us) server_negotiation.cc:299] Negotiation successful
0806 00:53:46.642623 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:53:46.643673 29609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:46.629383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43793 (local address 127.27.77.193:50955)
0806 00:53:46.630016 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:46.630077 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:46.631394 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:46.632135 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:46.632143 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:46.632504 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:46.633087 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:46.633102 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.634056 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.634060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:46.634940 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:46.634949 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.635626 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.635630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:46.635808 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:46.636464 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:46.636480 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:46.638491 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:46.640520 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:46.640526 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:46.640549 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:46.640909 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:46.641260 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:46.641274 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:46.641279 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:46.641459 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:46.641986 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:46.641992 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:46.642231 (+ 239us) client_negotiation.cc:769] Sending connection context
0806 00:53:46.642405 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 00:53:46.642677 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":249,"threads_started":1}
I20260806 00:53:46.643844 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 29572
I20260806 00:53:46.644021 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:53:46.644502 27959 external_mini_cluster.cc:1428] Setting key ce1a402e3b30c1d23702b3c2af10b49c
I20260806 00:53:46.647475 29753 heartbeater.cc:347] Connected to a master server at 127.27.77.254:43793
I20260806 00:53:46.647679 29753 heartbeater.cc:464] Registering TS with master...
I20260806 00:53:46.648154 29753 heartbeater.cc:511] Master 127.27.77.254:43793 requested a full tablet report, sending...
I20260806 00:53:46.649847 29472 ts_manager.cc:194] Registered new tserver with Master: ae4f2afdb7644fb3b21179127b0b956c (127.27.77.193:36265)
I20260806 00:53:46.651026 29472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:50955
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
I20260806 00:53:46.659719 29541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:46.647625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:44659 (local address 127.27.77.254:43793)
0806 00:53:46.647798 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:46.647802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:46.649603 (+ 1801us) server_negotiation.cc:407] Connection header received
0806 00:53:46.651822 (+ 2219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:46.651826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:46.651884 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:46.651992 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:46.653460 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.653971 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.654864 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.655100 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.655222 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:46.656476 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:46.656484 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:46.656487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:46.656510 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:46.657975 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:46.658430 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:46.658438 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:46.658439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:46.658512 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:46.658806 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:46.658810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:46.658812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:46.658991 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:46.659104 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:46.659391 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 00:53:46.659510 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 00:53:46.659739 29762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:46.647603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43793 (local address 127.27.77.193:44659)
0806 00:53:46.649511 (+ 1908us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:46.649524 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:46.651660 (+ 2136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:46.652028 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:46.652031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:46.653198 (+ 1167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:46.653317 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:46.653324 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.654112 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.654115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:46.654709 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:46.654718 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.655231 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.655235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:46.655329 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:46.655951 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:46.655963 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:46.656331 (+ 368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:46.658090 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:46.658094 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:46.658096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:46.658312 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:46.658608 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:46.658612 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:46.658614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:46.658711 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:46.659092 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:46.659096 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:46.659211 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 00:53:46.659333 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 00:53:46.659496 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1807,"thread_start_us":126,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:53:46.681037 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:43793
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:46.800160 29765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:46.800438 29765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:46.800484 29765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:46.804499 29765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:46.804563 29765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:46.804603 29765 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:53:46.804636 29765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:53:46.810336 29765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.77.254:43793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:46.811407 29765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:46.812482 29765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:46.822247 29775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:46.822265 29773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:46.822256 29772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:46.822669 29765 server_base.cc:1034] running on GCE node
I20260806 00:53:46.823069 29765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:46.823800 29765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:46.824985 29765 hybrid_clock.cc:648] HybridClock initialized: now 1785977626824942 us; error 60 us; skew 500 ppm
Aug 06 00:53:46 dist-test-slave-sb2z krb5kdc[29409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977626, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:46.827929 29765 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:53:46.828845 29765 webserver.cc:460] Webserver started at http://127.27.77.194:35639/ using document root <none> and password file <none>
I20260806 00:53:46.829377 29765 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:46.829437 29765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:46.829603 29765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:46.831669 29765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "d1cd6e40982040bebe866c1a00d067ce"
format_stamp: "Formatted at 2026-08-06 00:53:46 on dist-test-slave-sb2z"
server_key: "14417719bb31abd234e24933750cbd58"
server_key_iv: "b095e94bf81bba45cdb5a39fe15abf5c"
server_key_version: "encryptionkey@0"
I20260806 00:53:46.832216 29765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "d1cd6e40982040bebe866c1a00d067ce"
format_stamp: "Formatted at 2026-08-06 00:53:46 on dist-test-slave-sb2z"
server_key: "14417719bb31abd234e24933750cbd58"
server_key_iv: "b095e94bf81bba45cdb5a39fe15abf5c"
server_key_version: "encryptionkey@0"
I20260806 00:53:46.835986 29765 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:53:46.838550 29784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:46.840342 29765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:53:46.840476 29765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "d1cd6e40982040bebe866c1a00d067ce"
format_stamp: "Formatted at 2026-08-06 00:53:46 on dist-test-slave-sb2z"
server_key: "14417719bb31abd234e24933750cbd58"
server_key_iv: "b095e94bf81bba45cdb5a39fe15abf5c"
server_key_version: "encryptionkey@0"
I20260806 00:53:46.840603 29765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:46.853021 29765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:46.856518 29765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:46.856742 29765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:46.858335 29765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:53:46.858412 29765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:46.858475 29765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:53:46.858513 29765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:53:46 dist-test-slave-sb2z krb5kdc[29409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977626, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:46.870406 29765 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:40357
I20260806 00:53:46.870572 29958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:40357 every 8 connection(s)
I20260806 00:53:46.871435 29765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:53:46.873806 29541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:46.859278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:51529 (local address 127.27.77.254:43793)
0806 00:53:46.859430 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:46.859434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:46.860520 (+ 1086us) server_negotiation.cc:407] Connection header received
0806 00:53:46.861826 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:46.861829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:46.861894 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:46.862013 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:46.863706 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.864552 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.865919 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.866315 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.866454 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:46.869556 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:46.869560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:46.869563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:46.869594 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:46.871450 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:46.872212 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:46.872217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:46.872219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:46.872331 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:46.872676 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:46.872679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:46.872680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:46.872902 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:46.873067 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:46.873547 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 00:53:46.873674 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 00:53:46.874557 29799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:46.859805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43793 (local address 127.27.77.194:51529)
0806 00:53:46.860334 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:46.860381 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:46.861508 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:46.862215 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:46.862227 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:46.862747 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:46.863477 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:46.863504 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.864717 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.864721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:46.865729 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:46.865742 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.866523 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.866530 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:46.866808 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:46.867577 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:46.867604 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:46.869370 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:46.871660 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:46.871666 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:46.871681 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:46.872069 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:46.872429 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:46.872432 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:46.872433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:46.872564 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:46.873064 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:46.873068 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:46.873294 (+ 226us) client_negotiation.cc:769] Sending connection context
0806 00:53:46.873442 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 00:53:46.873719 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":153,"threads_started":1}
I20260806 00:53:46.875933 29959 heartbeater.cc:347] Connected to a master server at 127.27.77.254:43793
I20260806 00:53:46.876076 29959 heartbeater.cc:464] Registering TS with master...
I20260806 00:53:46.876663 29959 heartbeater.cc:511] Master 127.27.77.254:43793 requested a full tablet report, sending...
I20260806 00:53:46.877880 29472 ts_manager.cc:194] Registered new tserver with Master: d1cd6e40982040bebe866c1a00d067ce (127.27.77.194:40357)
I20260806 00:53:46.878615 29472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:51529
I20260806 00:53:46.879143 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 29765
I20260806 00:53:46.879279 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:53:46.879639 27959 external_mini_cluster.cc:1428] Setting key 3e6b5d33911b81f81ec863195f269772
I20260806 00:53:46.887434 29968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:46.878356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43793 (local address 127.27.77.194:59683)
0806 00:53:46.878701 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:46.878717 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:46.878816 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:46.879834 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:46.879837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:46.880053 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:46.880180 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:46.880188 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.881271 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.881274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:46.882042 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:46.882053 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.882690 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.882692 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:46.882758 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:46.883271 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:46.883293 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:46.883685 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:46.885800 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:46.885803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:46.885804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:46.885993 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:46.886320 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:46.886323 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:46.886324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:46.886400 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:46.886993 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:46.886995 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:46.887069 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 00:53:46.887171 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 00:53:46.887282 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":143,"threads_started":1}
I20260806 00:53:46.887845 29541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:46.878414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:59683 (local address 127.27.77.254:43793)
0806 00:53:46.878564 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:46.878569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:46.878779 (+ 210us) server_negotiation.cc:407] Connection header received
0806 00:53:46.879652 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:46.879659 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:46.879718 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:46.879815 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:46.880321 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.881081 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.882208 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:46.882502 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:46.882563 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:46.883832 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:46.883838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:46.883843 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:46.883879 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:46.885682 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:46.886098 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:46.886106 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:46.886111 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:46.886217 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:46.886505 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:46.886511 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:46.886516 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:46.886878 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:46.887022 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:46.887252 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 00:53:46.887403 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:53:46.910951 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:43793
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:47.034938 29973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:47.035224 29973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:47.035276 29973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:47.039898 29973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:47.039971 29973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:47.040030 29973 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:53:47.040072 29973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:53:47.046285 29973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.77.254:43793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:47.047540 29973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:47.048691 29973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:47.059058 29980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:47.059058 29982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:47.059222 29978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:47.059616 29973 server_base.cc:1034] running on GCE node
I20260806 00:53:47.060019 29973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:47.060654 29973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:47.061851 29973 hybrid_clock.cc:648] HybridClock initialized: now 1785977627061820 us; error 44 us; skew 500 ppm
Aug 06 00:53:47 dist-test-slave-sb2z krb5kdc[29409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977627, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:47.064764 29973 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:53:47.065685 29973 webserver.cc:460] Webserver started at http://127.27.77.195:36479/ using document root <none> and password file <none>
I20260806 00:53:47.066262 29973 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:47.066356 29973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:47.066597 29973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:47.068749 29973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "eb6f8153541a4c1c990d2ebe763caa49"
format_stamp: "Formatted at 2026-08-06 00:53:47 on dist-test-slave-sb2z"
server_key: "58c190fbe4ae99cf6298fb6ecdef119b"
server_key_iv: "9f47eb15cc8ca6cd7845cd10c77a1738"
server_key_version: "encryptionkey@0"
I20260806 00:53:47.069335 29973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "eb6f8153541a4c1c990d2ebe763caa49"
format_stamp: "Formatted at 2026-08-06 00:53:47 on dist-test-slave-sb2z"
server_key: "58c190fbe4ae99cf6298fb6ecdef119b"
server_key_iv: "9f47eb15cc8ca6cd7845cd10c77a1738"
server_key_version: "encryptionkey@0"
I20260806 00:53:47.073113 29973 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:53:47.075693 29994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:47.076956 29973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:53:47.077090 29973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "eb6f8153541a4c1c990d2ebe763caa49"
format_stamp: "Formatted at 2026-08-06 00:53:47 on dist-test-slave-sb2z"
server_key: "58c190fbe4ae99cf6298fb6ecdef119b"
server_key_iv: "9f47eb15cc8ca6cd7845cd10c77a1738"
server_key_version: "encryptionkey@0"
I20260806 00:53:47.077209 29973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:47.115294 29973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:47.118681 29973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:47.118921 29973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:47.120553 29973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:53:47.120631 29973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:47.120712 29973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:53:47.120757 29973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:53:47 dist-test-slave-sb2z krb5kdc[29409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977627, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:47.133201 29973 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:41927
I20260806 00:53:47.133327 30157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:41927 every 8 connection(s)
I20260806 00:53:47.134186 29973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:53:47.136343 29541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.121562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:38563 (local address 127.27.77.254:43793)
0806 00:53:47.121711 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.121715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.122895 (+ 1180us) server_negotiation.cc:407] Connection header received
0806 00:53:47.124217 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.124220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.124287 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.124420 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:47.126195 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.127024 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.128391 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.128767 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.128884 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.132232 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:47.132237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:47.132239 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:47.132277 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:47.133952 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:47.134581 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:47.134584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:47.134586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:47.134722 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:47.135185 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:47.135190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:47.135194 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:47.135442 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:47.135570 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.136102 (+ 532us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.136222 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:53:47.137199 30011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.122174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43793 (local address 127.27.77.195:38563)
0806 00:53:47.122730 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:47.122778 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:47.123909 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:47.124648 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:47.124661 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:47.125156 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:47.126011 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.126036 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.127178 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.127182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.128174 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.128186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.128962 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.128967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.129227 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.129944 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:47.129997 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:47.132036 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:47.134099 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:47.134105 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:47.134120 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:47.134464 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:47.134847 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:47.134850 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:47.134853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:47.135031 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:47.135614 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:47.135620 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:47.135865 (+ 245us) client_negotiation.cc:769] Sending connection context
0806 00:53:47.136022 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 00:53:47.136297 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":141,"threads_started":1}
I20260806 00:53:47.138798 30158 heartbeater.cc:347] Connected to a master server at 127.27.77.254:43793
I20260806 00:53:47.138969 30158 heartbeater.cc:464] Registering TS with master...
I20260806 00:53:47.139532 30158 heartbeater.cc:511] Master 127.27.77.254:43793 requested a full tablet report, sending...
I20260806 00:53:47.139942 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 29973
I20260806 00:53:47.140076 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:53:47.140378 27959 external_mini_cluster.cc:1428] Setting key 72ebbad1ce84b3e548b2d144e7c53bb1
I20260806 00:53:47.140703 29472 ts_manager.cc:194] Registered new tserver with Master: eb6f8153541a4c1c990d2ebe763caa49 (127.27.77.195:41927)
I20260806 00:53:47.141491 29472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:38563
I20260806 00:53:47.142630 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:53:47.150271 29541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.141256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:36519 (local address 127.27.77.254:43793)
0806 00:53:47.141369 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.141373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.141538 (+ 165us) server_negotiation.cc:407] Connection header received
0806 00:53:47.141644 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.141649 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.141698 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.141775 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:47.142294 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.143048 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.144309 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.144703 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.144830 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.146419 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:47.146423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:47.146425 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:47.146472 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:47.148238 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:47.148811 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:47.148818 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:47.148822 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:47.148927 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:47.149265 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:47.149269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:47.149272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:47.149524 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:47.149633 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.150038 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.150155 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 00:53:47.150699 30166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.141118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43793 (local address 127.27.77.195:36519)
0806 00:53:47.141442 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:47.141457 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:47.141559 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:47.141898 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:47.141901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:47.142079 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:47.142176 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.142181 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.143204 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.143208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.144135 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.144147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.144852 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.144855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.144978 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.145727 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:47.145746 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:47.146254 (+ 508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:47.148362 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:47.148365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:47.148366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:47.148585 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:47.149079 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:47.149081 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:47.149082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:47.149162 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:47.149635 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:47.149637 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:47.149798 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 00:53:47.149927 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 00:53:47.150161 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":105,"threads_started":1}
Aug 06 00:53:47 dist-test-slave-sb2z krb5kdc[29409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977627, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 00:53:47 dist-test-slave-sb2z krb5kdc[29409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977627, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:47.173014 29541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.164395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42440 (local address 127.27.77.254:43793)
0806 00:53:47.164543 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.164548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.164565 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:53:47.164615 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.164617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.164675 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.164747 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:47.165429 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.165951 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.166852 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.167074 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.167155 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.169774 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:47.169778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:47.169780 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:47.169809 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:47.171332 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:47.171912 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:47.171915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:47.171916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:47.171997 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:47.172274 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:47.172276 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:47.172277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:47.172456 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:47.172560 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.172789 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.172888 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 00:53:47.176666 29471 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:53:47.180080 29471 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:53:47.196123 30177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.189501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:40357 (local address 127.0.0.1:60564)
0806 00:53:47.189874 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:47.189929 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:47.190072 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:47.191650 (+ 1578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:47.191656 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:47.191710 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:47.191842 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.191848 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.193259 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.193263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.194544 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.194555 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.195661 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.195665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.195786 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.195819 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 00:53:47.195892 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 00:53:47.195965 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":27,"spinlock_wait_cycles":3456,"thread_start_us":91,"threads_started":1}
I20260806 00:53:47.196195 30180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.190105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60564 (local address 127.27.77.194:40357)
0806 00:53:47.191085 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.191098 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.191121 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:53:47.191199 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.191203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.191382 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.191572 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:47.191963 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.192947 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.194677 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.195502 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.195663 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.195829 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.195921 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.196033 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869,"thread_start_us":114,"threads_started":1}
I20260806 00:53:47.196463 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.190402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43618 (local address 127.27.77.193:36265)
0806 00:53:47.190858 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.190865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.190892 (+ 27us) server_negotiation.cc:407] Connection header received
0806 00:53:47.191049 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.191056 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.191221 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.191340 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:47.191768 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.192858 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.194403 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.195166 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.195281 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.195428 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.196059 (+ 631us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.196166 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":123,"threads_started":1}
I20260806 00:53:47.195945 30181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.189707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:36265 (local address 127.0.0.1:43618)
0806 00:53:47.190708 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:47.190723 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:47.190822 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:47.191382 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:47.191388 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:47.191412 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:47.191605 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.191613 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.192977 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.192980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.194274 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.194284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.195300 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.195304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.195413 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.195455 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 00:53:47.195606 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 00:53:47.195759 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"mutex_wait_us":24,"thread_start_us":106,"threads_started":1}
I20260806 00:53:47.197796 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.190234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:41927 (local address 127.0.0.1:39886)
0806 00:53:47.191588 (+ 1354us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:47.191601 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:47.191719 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:47.192318 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:47.192321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:47.192338 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:47.192446 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.192453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.193832 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.193836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.194863 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.194871 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.196116 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.196119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.196215 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.197478 (+ 1263us) client_negotiation.cc:769] Sending connection context
0806 00:53:47.197591 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 00:53:47.197668 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1266,"spinlock_wait_cycles":50432,"thread_start_us":83,"threads_started":1}
I20260806 00:53:47.197973 30187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.190644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39886 (local address 127.27.77.195:41927)
0806 00:53:47.191085 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.191098 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.191758 (+ 660us) server_negotiation.cc:407] Connection header received
0806 00:53:47.191841 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.191846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.192035 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.192182 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:47.192592 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.193691 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.194992 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.195776 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.195878 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.196050 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.197632 (+ 1582us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.197730 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":116,"threads_started":1}
I20260806 00:53:47.199339 29859 tablet_service.cc:1467] Processing CreateTablet for tablet afc5a3367c28438293c1884a6a8aec1d (DEFAULT_TABLE table=test-table [id=b115e369fc1549588cf5a4e6a8697e71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:47.199337 29666 tablet_service.cc:1467] Processing CreateTablet for tablet afc5a3367c28438293c1884a6a8aec1d (DEFAULT_TABLE table=test-table [id=b115e369fc1549588cf5a4e6a8697e71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:47.200573 29859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afc5a3367c28438293c1884a6a8aec1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:47.200579 29666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afc5a3367c28438293c1884a6a8aec1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:47.202378 30065 tablet_service.cc:1467] Processing CreateTablet for tablet afc5a3367c28438293c1884a6a8aec1d (DEFAULT_TABLE table=test-table [id=b115e369fc1549588cf5a4e6a8697e71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:47.203270 30065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afc5a3367c28438293c1884a6a8aec1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:47.209197 30189 tablet_bootstrap.cc:492] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce: Bootstrap starting.
I20260806 00:53:47.209518 30188 tablet_bootstrap.cc:492] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c: Bootstrap starting.
I20260806 00:53:47.210278 30190 tablet_bootstrap.cc:492] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49: Bootstrap starting.
I20260806 00:53:47.211681 30189 tablet_bootstrap.cc:654] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:47.211822 30188 tablet_bootstrap.cc:654] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:47.212419 30190 tablet_bootstrap.cc:654] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:47.212582 30188 log.cc:824] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:47.212582 30189 log.cc:824] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:47.213209 30190 log.cc:824] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:47.214579 30188 tablet_bootstrap.cc:492] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c: No bootstrap required, opened a new log
I20260806 00:53:47.214795 30188 ts_tablet_manager.cc:1397] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260806 00:53:47.215131 30190 tablet_bootstrap.cc:492] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49: No bootstrap required, opened a new log
I20260806 00:53:47.215322 30190 ts_tablet_manager.cc:1397] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 00:53:47.218321 30190 raft_consensus.cc:348] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.218389 30188 raft_consensus.cc:348] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.218569 30190 raft_consensus.cc:374] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:47.218609 30188 raft_consensus.cc:374] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:47.218640 30190 raft_consensus.cc:729] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6f8153541a4c1c990d2ebe763caa49, State: Initialized, Role: FOLLOWER
I20260806 00:53:47.218673 30188 raft_consensus.cc:729] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4f2afdb7644fb3b21179127b0b956c, State: Initialized, Role: FOLLOWER
I20260806 00:53:47.219142 30188 consensus_queue.cc:260] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.219147 30190 consensus_queue.cc:260] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.219769 29753 heartbeater.cc:503] Master 127.27.77.254:43793 was elected leader, sending a full tablet report...
I20260806 00:53:47.220082 30189 tablet_bootstrap.cc:492] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce: No bootstrap required, opened a new log
I20260806 00:53:47.220178 30190 ts_tablet_manager.cc:1428] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260806 00:53:47.220265 30189 ts_tablet_manager.cc:1397] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.004s
I20260806 00:53:47.220518 30158 heartbeater.cc:503] Master 127.27.77.254:43793 was elected leader, sending a full tablet report...
I20260806 00:53:47.221199 30188 ts_tablet_manager.cc:1428] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:53:47.224260 30189 raft_consensus.cc:348] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.224550 30189 raft_consensus.cc:374] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:47.224629 30189 raft_consensus.cc:729] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1cd6e40982040bebe866c1a00d067ce, State: Initialized, Role: FOLLOWER
I20260806 00:53:47.225021 30189 consensus_queue.cc:260] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.225621 29959 heartbeater.cc:503] Master 127.27.77.254:43793 was elected leader, sending a full tablet report...
I20260806 00:53:47.225925 30189 ts_tablet_manager.cc:1428] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:53:47.351915 30196 raft_consensus.cc:482] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:53:47.352154 30196 raft_consensus.cc:504] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.353358 30196 leader_election.cc:288] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae4f2afdb7644fb3b21179127b0b956c (127.27.77.193:36265), eb6f8153541a4c1c990d2ebe763caa49 (127.27.77.195:41927)
I20260806 00:53:47.358114 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.353850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:60447 (local address 127.27.77.193:36265)
0806 00:53:47.354044 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.354047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.354062 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:53:47.354113 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.354115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.354155 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.354254 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:47.354604 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.355202 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.356810 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.357497 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.357634 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.357690 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.357906 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.357990 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 00:53:47.358120 29799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.353581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:36265 (local address 127.27.77.194:60447)
0806 00:53:47.353711 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:47.353731 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:47.353833 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:47.354268 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:47.354271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:47.354292 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:47.354432 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.354438 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.355318 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.355323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.356650 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.356662 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.357635 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.357638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.357758 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.357786 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 00:53:47.357878 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 00:53:47.357959 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 00:53:47.358665 30187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.353840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:43247 (local address 127.27.77.195:41927)
0806 00:53:47.353981 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.353986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.354179 (+ 193us) server_negotiation.cc:407] Connection header received
0806 00:53:47.354328 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.354331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.354405 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.354512 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:47.354844 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.355560 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.357301 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.357813 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.357926 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.357984 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.358439 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.358527 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 00:53:47.358925 29693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc5a3367c28438293c1884a6a8aec1d" candidate_uuid: "d1cd6e40982040bebe866c1a00d067ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae4f2afdb7644fb3b21179127b0b956c" is_pre_election: true
I20260806 00:53:47.359048 30199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.353699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:41927 (local address 127.27.77.194:43247)
0806 00:53:47.354084 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:47.354095 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:47.354216 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:47.354521 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:47.354526 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:47.354563 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:47.354716 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.354722 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.355695 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.355698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.357170 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.357179 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.358044 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.358048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.358152 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.358321 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 00:53:47.358814 (+ 493us) client_negotiation.cc:240] Negotiation successful
0806 00:53:47.358889 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":133,"threads_started":1}
I20260806 00:53:47.359278 29693 raft_consensus.cc:2454] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1cd6e40982040bebe866c1a00d067ce in term 0.
I20260806 00:53:47.359679 30098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc5a3367c28438293c1884a6a8aec1d" candidate_uuid: "d1cd6e40982040bebe866c1a00d067ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6f8153541a4c1c990d2ebe763caa49" is_pre_election: true
I20260806 00:53:47.359889 29788 leader_election.cc:302] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae4f2afdb7644fb3b21179127b0b956c, d1cd6e40982040bebe866c1a00d067ce; no voters:
I20260806 00:53:47.359975 30098 raft_consensus.cc:2454] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1cd6e40982040bebe866c1a00d067ce in term 0.
I20260806 00:53:47.360184 30196 raft_consensus.cc:2781] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:53:47.360286 30196 raft_consensus.cc:482] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:53:47.360339 30196 raft_consensus.cc:3037] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:47.361490 30196 raft_consensus.cc:504] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.361923 30196 leader_election.cc:288] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [CANDIDATE]: Term 1 election: Requested vote from peers ae4f2afdb7644fb3b21179127b0b956c (127.27.77.193:36265), eb6f8153541a4c1c990d2ebe763caa49 (127.27.77.195:41927)
I20260806 00:53:47.362274 30098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc5a3367c28438293c1884a6a8aec1d" candidate_uuid: "d1cd6e40982040bebe866c1a00d067ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6f8153541a4c1c990d2ebe763caa49"
I20260806 00:53:47.362280 29693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc5a3367c28438293c1884a6a8aec1d" candidate_uuid: "d1cd6e40982040bebe866c1a00d067ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae4f2afdb7644fb3b21179127b0b956c"
I20260806 00:53:47.362432 30098 raft_consensus.cc:3037] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:47.362437 29693 raft_consensus.cc:3037] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:47.363744 29693 raft_consensus.cc:2454] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1cd6e40982040bebe866c1a00d067ce in term 1.
I20260806 00:53:47.364090 30098 raft_consensus.cc:2454] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1cd6e40982040bebe866c1a00d067ce in term 1.
I20260806 00:53:47.364125 29788 leader_election.cc:302] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae4f2afdb7644fb3b21179127b0b956c, d1cd6e40982040bebe866c1a00d067ce; no voters:
I20260806 00:53:47.364292 30196 raft_consensus.cc:2781] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:47.364591 30196 raft_consensus.cc:686] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [term 1 LEADER]: Becoming Leader. State: Replica: d1cd6e40982040bebe866c1a00d067ce, State: Running, Role: LEADER
I20260806 00:53:47.364929 30196 consensus_queue.cc:237] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.368075 29471 catalog_manager.cc:5495] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce reported cstate change: term changed from 0 to 1, leader changed from <none> to d1cd6e40982040bebe866c1a00d067ce (127.27.77.194). New cstate: current_term: 1 leader_uuid: "d1cd6e40982040bebe866c1a00d067ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } health_report { overall_health: UNKNOWN } } }
W20260806 00:53:47.372742 29961 tablet.cc:2369] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:53:47.385344 30160 tablet.cc:2369] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:53:47.392532 29755 tablet.cc:2369] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:53:47.394264 30180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.390310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60578 (local address 127.27.77.194:40357)
0806 00:53:47.390442 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.390445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.390459 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:53:47.390689 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.390692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.390752 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.390868 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:47.391211 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.391758 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.392708 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.392930 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.393044 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.393402 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:47.393807 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:47.393940 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.394057 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.394120 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:53:47.399911 29693 raft_consensus.cc:1261] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c [term 1 FOLLOWER]: Refusing update from remote peer d1cd6e40982040bebe866c1a00d067ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:47.399919 30098 raft_consensus.cc:1261] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49 [term 1 FOLLOWER]: Refusing update from remote peer d1cd6e40982040bebe866c1a00d067ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:47.400612 30200 consensus_queue.cc:1035] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:47.400812 30196 consensus_queue.cc:1035] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:47.411657 30205 mvcc.cc:205] Tried to move back new op lower bound from 7315364361822662656 to 7315364361689567232. Current Snapshot: MvccSnapshot[applied={T|T < 7315364361822662656}]
I20260806 00:53:47.411861 30206 mvcc.cc:205] Tried to move back new op lower bound from 7315364361822662656 to 7315364361689567232. Current Snapshot: MvccSnapshot[applied={T|T < 7315364361822662656}]
I20260806 00:53:47.421222 29470 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:53:47.421388 29470 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:53:47.423974 29470 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08: Sending DeleteTablet for 3 replicas of tablet afc5a3367c28438293c1884a6a8aec1d
I20260806 00:53:47.424790 29859 tablet_service.cc:1514] Processing DeleteTablet for tablet afc5a3367c28438293c1884a6a8aec1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:47 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:60564
I20260806 00:53:47.424786 29666 tablet_service.cc:1514] Processing DeleteTablet for tablet afc5a3367c28438293c1884a6a8aec1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:47 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:43618
I20260806 00:53:47.425324 30216 tablet_replica.cc:330] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce: stopping tablet replica
I20260806 00:53:47.425347 30065 tablet_service.cc:1514] Processing DeleteTablet for tablet afc5a3367c28438293c1884a6a8aec1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:47 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:39886
I20260806 00:53:47.425766 30216 raft_consensus.cc:2229] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:53:47.425837 30218 tablet_replica.cc:330] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49: stopping tablet replica
I20260806 00:53:47.426218 30217 tablet_replica.cc:330] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c: stopping tablet replica
I20260806 00:53:47.426186 29470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 00:53:47.426508 30216 raft_consensus.cc:2258] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:47.426505 30217 raft_consensus.cc:2229] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 00:53:47.426614 29470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:53:47.426823 30217 raft_consensus.cc:2258] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:47.427410 30218 raft_consensus.cc:2229] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:53:47.427731 30218 raft_consensus.cc:2258] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:47.428530 30216 ts_tablet_manager.cc:1905] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:47.429051 30217 ts_tablet_manager.cc:1905] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:47.430871 30218 ts_tablet_manager.cc:1905] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:47.431777 30216 ts_tablet_manager.cc:1918] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:53:47.431945 30216 log.cc:1196] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/afc5a3367c28438293c1884a6a8aec1d
I20260806 00:53:47.431962 30217 ts_tablet_manager.cc:1918] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:53:47.432065 30217 log.cc:1196] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/wal/wals/afc5a3367c28438293c1884a6a8aec1d
I20260806 00:53:47.432441 30217 ts_tablet_manager.cc:1939] T afc5a3367c28438293c1884a6a8aec1d P ae4f2afdb7644fb3b21179127b0b956c: Deleting consensus metadata
I20260806 00:53:47.432605 30216 ts_tablet_manager.cc:1939] T afc5a3367c28438293c1884a6a8aec1d P d1cd6e40982040bebe866c1a00d067ce: Deleting consensus metadata
I20260806 00:53:47.432819 29666 tablet_service.cc:1467] Processing CreateTablet for tablet ac7bc7e4c9754f5d921c89da8f371ff9 (DEFAULT_TABLE table=test-table [id=33804d24076d4d0b84a4030523dc025f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:47.433104 29666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac7bc7e4c9754f5d921c89da8f371ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:47.433773 30218 ts_tablet_manager.cc:1918] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:53:47.433881 30218 log.cc:1196] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/afc5a3367c28438293c1884a6a8aec1d
I20260806 00:53:47.434207 30218 ts_tablet_manager.cc:1939] T afc5a3367c28438293c1884a6a8aec1d P eb6f8153541a4c1c990d2ebe763caa49: Deleting consensus metadata
I20260806 00:53:47.435173 30188 tablet_bootstrap.cc:492] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c: Bootstrap starting.
I20260806 00:53:47.435447 29455 catalog_manager.cc:4841] TS ae4f2afdb7644fb3b21179127b0b956c (127.27.77.193:36265): tablet afc5a3367c28438293c1884a6a8aec1d (table test-table [id=b115e369fc1549588cf5a4e6a8697e71]) successfully deleted
I20260806 00:53:47.435953 29458 catalog_manager.cc:4841] TS eb6f8153541a4c1c990d2ebe763caa49 (127.27.77.195:41927): tablet afc5a3367c28438293c1884a6a8aec1d (table test-table [id=b115e369fc1549588cf5a4e6a8697e71]) successfully deleted
I20260806 00:53:47.436110 29458 catalog_manager.cc:4841] TS d1cd6e40982040bebe866c1a00d067ce (127.27.77.194:40357): tablet afc5a3367c28438293c1884a6a8aec1d (table test-table [id=b115e369fc1549588cf5a4e6a8697e71]) successfully deleted
I20260806 00:53:47.436268 30188 tablet_bootstrap.cc:654] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:47.436414 29859 tablet_service.cc:1467] Processing CreateTablet for tablet ac7bc7e4c9754f5d921c89da8f371ff9 (DEFAULT_TABLE table=test-table [id=33804d24076d4d0b84a4030523dc025f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:47.436612 30065 tablet_service.cc:1467] Processing CreateTablet for tablet ac7bc7e4c9754f5d921c89da8f371ff9 (DEFAULT_TABLE table=test-table [id=33804d24076d4d0b84a4030523dc025f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:47.436823 29859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac7bc7e4c9754f5d921c89da8f371ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:47.436861 30065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac7bc7e4c9754f5d921c89da8f371ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:47.438623 30190 tablet_bootstrap.cc:492] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49: Bootstrap starting.
I20260806 00:53:47.439728 30190 tablet_bootstrap.cc:654] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:47.440267 30188 tablet_bootstrap.cc:492] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c: No bootstrap required, opened a new log
I20260806 00:53:47.440369 30188 ts_tablet_manager.cc:1397] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 00:53:47.440665 30189 tablet_bootstrap.cc:492] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce: Bootstrap starting.
I20260806 00:53:47.441123 30190 tablet_bootstrap.cc:492] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49: No bootstrap required, opened a new log
I20260806 00:53:47.441201 30190 ts_tablet_manager.cc:1397] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:53:47.441289 30188 raft_consensus.cc:348] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.441434 30188 raft_consensus.cc:374] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:47.441470 30188 raft_consensus.cc:729] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4f2afdb7644fb3b21179127b0b956c, State: Initialized, Role: FOLLOWER
I20260806 00:53:47.441609 30188 consensus_queue.cc:260] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.441675 30190 raft_consensus.cc:348] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.441776 30190 raft_consensus.cc:374] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:47.441807 30190 raft_consensus.cc:729] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6f8153541a4c1c990d2ebe763caa49, State: Initialized, Role: FOLLOWER
I20260806 00:53:47.441920 30190 consensus_queue.cc:260] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.442178 30190 ts_tablet_manager.cc:1428] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:53:47.442174 30189 tablet_bootstrap.cc:654] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:47.443092 30188 ts_tablet_manager.cc:1428] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:53:47.445909 30189 tablet_bootstrap.cc:492] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce: No bootstrap required, opened a new log
I20260806 00:53:47.446239 30189 ts_tablet_manager.cc:1397] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260806 00:53:47.446990 30189 raft_consensus.cc:348] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.447175 30189 raft_consensus.cc:374] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:47.447237 30189 raft_consensus.cc:729] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1cd6e40982040bebe866c1a00d067ce, State: Initialized, Role: FOLLOWER
I20260806 00:53:47.447427 30189 consensus_queue.cc:260] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.448328 30189 ts_tablet_manager.cc:1428] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:53:47.461268 30195 raft_consensus.cc:482] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:53:47.461436 30195 raft_consensus.cc:504] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.462809 30195 leader_election.cc:288] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae4f2afdb7644fb3b21179127b0b956c (127.27.77.193:36265), d1cd6e40982040bebe866c1a00d067ce (127.27.77.194:40357)
I20260806 00:53:47.468218 30180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.463197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:42641 (local address 127.27.77.194:40357)
0806 00:53:47.463324 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.463331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.463348 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:53:47.463558 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.463562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.463643 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.463725 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:47.464235 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.465318 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.466779 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.467411 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.467572 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.467630 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.467995 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.468114 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:53:47.468329 30011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.463074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:40357 (local address 127.27.77.195:42641)
0806 00:53:47.463279 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:47.463295 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:47.463391 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:47.463908 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:47.463911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:47.463931 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:47.464069 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.464076 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.465463 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.465466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.466629 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.466635 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.467586 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.467590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.467792 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.467832 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 00:53:47.467964 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 00:53:47.468076 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":55}
I20260806 00:53:47.469074 29886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac7bc7e4c9754f5d921c89da8f371ff9" candidate_uuid: "eb6f8153541a4c1c990d2ebe763caa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1cd6e40982040bebe866c1a00d067ce" is_pre_election: true
I20260806 00:53:47.469211 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.463816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:53861 (local address 127.27.77.193:36265)
0806 00:53:47.463984 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.463988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.464002 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:53:47.464058 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.464061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.464110 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.464184 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:47.464525 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.465027 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.466782 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.467411 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.467578 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.467630 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.468976 (+ 1346us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.469084 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 00:53:47.469312 30223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.463152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:36265 (local address 127.27.77.195:53861)
0806 00:53:47.463593 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:47.463607 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:47.463705 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:47.464292 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:47.464294 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:47.464308 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:47.464404 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.464408 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.465154 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.465158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.466622 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.466635 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.468560 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.468568 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.468768 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.468816 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 00:53:47.468956 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 00:53:47.469061 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":58,"thread_start_us":168,"threads_started":1}
I20260806 00:53:47.469348 29886 raft_consensus.cc:2454] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb6f8153541a4c1c990d2ebe763caa49 in term 0.
I20260806 00:53:47.469863 30005 leader_election.cc:302] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1cd6e40982040bebe866c1a00d067ce, eb6f8153541a4c1c990d2ebe763caa49; no voters:
I20260806 00:53:47.469925 29693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac7bc7e4c9754f5d921c89da8f371ff9" candidate_uuid: "eb6f8153541a4c1c990d2ebe763caa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae4f2afdb7644fb3b21179127b0b956c" is_pre_election: true
I20260806 00:53:47.470073 29693 raft_consensus.cc:2454] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb6f8153541a4c1c990d2ebe763caa49 in term 0.
I20260806 00:53:47.470170 30195 raft_consensus.cc:2781] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:53:47.470218 30195 raft_consensus.cc:482] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:53:47.470261 30195 raft_consensus.cc:3037] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:47.471421 30195 raft_consensus.cc:504] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.471850 30195 leader_election.cc:288] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [CANDIDATE]: Term 1 election: Requested vote from peers ae4f2afdb7644fb3b21179127b0b956c (127.27.77.193:36265), d1cd6e40982040bebe866c1a00d067ce (127.27.77.194:40357)
I20260806 00:53:47.472216 29886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac7bc7e4c9754f5d921c89da8f371ff9" candidate_uuid: "eb6f8153541a4c1c990d2ebe763caa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1cd6e40982040bebe866c1a00d067ce"
I20260806 00:53:47.472362 29886 raft_consensus.cc:3037] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:47.472383 29693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac7bc7e4c9754f5d921c89da8f371ff9" candidate_uuid: "eb6f8153541a4c1c990d2ebe763caa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae4f2afdb7644fb3b21179127b0b956c"
I20260806 00:53:47.472508 29693 raft_consensus.cc:3037] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:47.473724 29693 raft_consensus.cc:2454] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6f8153541a4c1c990d2ebe763caa49 in term 1.
I20260806 00:53:47.473852 29886 raft_consensus.cc:2454] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6f8153541a4c1c990d2ebe763caa49 in term 1.
I20260806 00:53:47.474133 30000 leader_election.cc:302] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae4f2afdb7644fb3b21179127b0b956c, eb6f8153541a4c1c990d2ebe763caa49; no voters:
I20260806 00:53:47.474344 30195 raft_consensus.cc:2781] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:47.474561 30195 raft_consensus.cc:686] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [term 1 LEADER]: Becoming Leader. State: Replica: eb6f8153541a4c1c990d2ebe763caa49, State: Running, Role: LEADER
I20260806 00:53:47.474818 30195 consensus_queue.cc:237] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.477385 29470 catalog_manager.cc:5495] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb6f8153541a4c1c990d2ebe763caa49 (127.27.77.195). New cstate: current_term: 1 leader_uuid: "eb6f8153541a4c1c990d2ebe763caa49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } health_report { overall_health: HEALTHY } } }
Aug 06 00:53:47 dist-test-slave-sb2z krb5kdc[29409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977626, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:47.514071 30238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.505325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43793 (local address 127.0.0.1:42442)
0806 00:53:47.505646 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:47.505658 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:47.505798 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:47.506047 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:47.506058 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:47.506584 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:47.506705 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.506716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.507487 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.507492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.508067 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.508075 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.508513 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.508516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.508590 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.509231 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:47.509243 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:47.510656 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:47.512363 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:47.512375 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:47.512381 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:47.512616 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:47.512893 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:47.512898 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:47.512900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:47.512985 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:47.513388 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:47.513395 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:47.513512 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 00:53:47.513665 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 00:53:47.513865 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":112,"threads_started":1}
I20260806 00:53:47.514273 29541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.505417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42442 (local address 127.27.77.254:43793)
0806 00:53:47.505576 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.505582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.505736 (+ 154us) server_negotiation.cc:407] Connection header received
0806 00:53:47.505895 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.505898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.505942 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.506030 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:47.506840 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.507374 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.508191 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.508402 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.508512 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.510790 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:47.510794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:47.510799 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:47.510825 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:47.512255 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:47.512718 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:47.512724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:47.512725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:47.512801 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:47.513092 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:47.513096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:47.513097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:47.513271 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:47.513390 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.513717 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.514035 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":46}
I20260806 00:53:47.518716 29470 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:42442:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 00:53:47.519508 29470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:53:47.524839 29666 tablet_service.cc:1467] Processing CreateTablet for tablet 67209770726b4cd5ac0f38091ecf685f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=83c2f7a8bc694923965c1cf2854752cd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:53:47.525146 29859 tablet_service.cc:1467] Processing CreateTablet for tablet 67209770726b4cd5ac0f38091ecf685f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=83c2f7a8bc694923965c1cf2854752cd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:53:47.525326 29666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67209770726b4cd5ac0f38091ecf685f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:47.525204 30065 tablet_service.cc:1467] Processing CreateTablet for tablet 67209770726b4cd5ac0f38091ecf685f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=83c2f7a8bc694923965c1cf2854752cd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:53:47.525548 29859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67209770726b4cd5ac0f38091ecf685f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:47.525588 30065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67209770726b4cd5ac0f38091ecf685f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:47.528626 30190 tablet_bootstrap.cc:492] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49: Bootstrap starting.
I20260806 00:53:47.528626 30188 tablet_bootstrap.cc:492] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c: Bootstrap starting.
I20260806 00:53:47.528808 30189 tablet_bootstrap.cc:492] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce: Bootstrap starting.
I20260806 00:53:47.529743 30188 tablet_bootstrap.cc:654] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:47.529770 30190 tablet_bootstrap.cc:654] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:47.529917 30189 tablet_bootstrap.cc:654] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:47.531734 30189 tablet_bootstrap.cc:492] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce: No bootstrap required, opened a new log
I20260806 00:53:47.531754 30190 tablet_bootstrap.cc:492] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49: No bootstrap required, opened a new log
I20260806 00:53:47.531827 30189 ts_tablet_manager.cc:1397] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:53:47.531844 30190 ts_tablet_manager.cc:1397] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260806 00:53:47.532341 30189 raft_consensus.cc:348] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.532469 30189 raft_consensus.cc:374] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:47.532380 30190 raft_consensus.cc:348] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.532505 30189 raft_consensus.cc:729] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1cd6e40982040bebe866c1a00d067ce, State: Initialized, Role: FOLLOWER
I20260806 00:53:47.532541 30190 raft_consensus.cc:374] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:47.532585 30190 raft_consensus.cc:729] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6f8153541a4c1c990d2ebe763caa49, State: Initialized, Role: FOLLOWER
I20260806 00:53:47.532630 30189 consensus_queue.cc:260] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.532724 30188 tablet_bootstrap.cc:492] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c: No bootstrap required, opened a new log
I20260806 00:53:47.532747 30190 consensus_queue.cc:260] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.532804 30188 ts_tablet_manager.cc:1397] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 00:53:47.532902 30189 ts_tablet_manager.cc:1428] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:53:47.532997 30195 tablet_replica.cc:439] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } } }
I20260806 00:53:47.533142 30195 tablet_replica.cc:442] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:53:47.533223 30209 tablet_replica.cc:439] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } } }
I20260806 00:53:47.533360 30209 tablet_replica.cc:442] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:53:47.533300 30188 raft_consensus.cc:348] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.533408 30188 raft_consensus.cc:374] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:47.533442 30188 raft_consensus.cc:729] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4f2afdb7644fb3b21179127b0b956c, State: Initialized, Role: FOLLOWER
I20260806 00:53:47.533564 30188 consensus_queue.cc:260] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.533912 30188 ts_tablet_manager.cc:1428] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:53:47.534193 30190 ts_tablet_manager.cc:1428] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 00:53:47.534126 30194 tablet_replica.cc:439] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } } }
I20260806 00:53:47.534276 30194 tablet_replica.cc:442] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:53:47.667968 30209 raft_consensus.cc:482] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:53:47.668128 30209 raft_consensus.cc:504] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.668670 30209 leader_election.cc:288] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae4f2afdb7644fb3b21179127b0b956c (127.27.77.193:36265), eb6f8153541a4c1c990d2ebe763caa49 (127.27.77.195:41927)
I20260806 00:53:47.669170 29693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67209770726b4cd5ac0f38091ecf685f" candidate_uuid: "d1cd6e40982040bebe866c1a00d067ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae4f2afdb7644fb3b21179127b0b956c" is_pre_election: true
I20260806 00:53:47.669181 30098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67209770726b4cd5ac0f38091ecf685f" candidate_uuid: "d1cd6e40982040bebe866c1a00d067ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6f8153541a4c1c990d2ebe763caa49" is_pre_election: true
I20260806 00:53:47.669364 29693 raft_consensus.cc:2454] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1cd6e40982040bebe866c1a00d067ce in term 0.
I20260806 00:53:47.669369 30098 raft_consensus.cc:2454] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1cd6e40982040bebe866c1a00d067ce in term 0.
I20260806 00:53:47.669736 29792 leader_election.cc:302] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1cd6e40982040bebe866c1a00d067ce, eb6f8153541a4c1c990d2ebe763caa49; no voters:
I20260806 00:53:47.669930 30209 raft_consensus.cc:2781] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:53:47.669997 30209 raft_consensus.cc:482] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:53:47.670038 30209 raft_consensus.cc:3037] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:47.671103 30209 raft_consensus.cc:504] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.671430 30209 leader_election.cc:288] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [CANDIDATE]: Term 1 election: Requested vote from peers ae4f2afdb7644fb3b21179127b0b956c (127.27.77.193:36265), eb6f8153541a4c1c990d2ebe763caa49 (127.27.77.195:41927)
I20260806 00:53:47.671830 29693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67209770726b4cd5ac0f38091ecf685f" candidate_uuid: "d1cd6e40982040bebe866c1a00d067ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae4f2afdb7644fb3b21179127b0b956c"
I20260806 00:53:47.671845 30098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67209770726b4cd5ac0f38091ecf685f" candidate_uuid: "d1cd6e40982040bebe866c1a00d067ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6f8153541a4c1c990d2ebe763caa49"
I20260806 00:53:47.671973 29693 raft_consensus.cc:3037] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:47.671984 30098 raft_consensus.cc:3037] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:47.672892 29693 raft_consensus.cc:2454] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1cd6e40982040bebe866c1a00d067ce in term 1.
I20260806 00:53:47.672936 30098 raft_consensus.cc:2454] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1cd6e40982040bebe866c1a00d067ce in term 1.
I20260806 00:53:47.673209 29788 leader_election.cc:302] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae4f2afdb7644fb3b21179127b0b956c, d1cd6e40982040bebe866c1a00d067ce; no voters:
I20260806 00:53:47.673411 30209 raft_consensus.cc:2781] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:47.673508 30209 raft_consensus.cc:686] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [term 1 LEADER]: Becoming Leader. State: Replica: d1cd6e40982040bebe866c1a00d067ce, State: Running, Role: LEADER
I20260806 00:53:47.673652 30209 consensus_queue.cc:237] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } }
I20260806 00:53:47.674403 30200 tablet_replica.cc:439] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce: TxnStatusTablet state changed. Reason: New leader d1cd6e40982040bebe866c1a00d067ce. Latest consensus state: current_term: 1 leader_uuid: "d1cd6e40982040bebe866c1a00d067ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } } }
I20260806 00:53:47.674512 30200 tablet_replica.cc:442] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:53:47.674873 30245 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 00:53:47.675135 30245 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 00:53:47.675719 29470 catalog_manager.cc:5495] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce reported cstate change: term changed from 0 to 1, leader changed from <none> to d1cd6e40982040bebe866c1a00d067ce (127.27.77.194). New cstate: current_term: 1 leader_uuid: "d1cd6e40982040bebe866c1a00d067ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } health_report { overall_health: UNKNOWN } } }
I20260806 00:53:47.694471 30180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.690653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60592 (local address 127.27.77.194:40357)
0806 00:53:47.690782 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.690785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.690881 (+ 96us) server_negotiation.cc:407] Connection header received
0806 00:53:47.691090 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.691092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.691139 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.691206 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:47.691485 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.692131 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.693354 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.693581 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.693680 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.694001 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:47.694104 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:47.694158 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.694304 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.694355 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:53:47.694511 30238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.690536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:40357 (local address 127.0.0.1:60592)
0806 00:53:47.690804 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:47.690819 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:47.690938 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:47.691256 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:47.691259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:47.691267 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:53:47.691388 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.691394 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.692243 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.692246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.693221 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.693232 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.693699 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.693703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.693801 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.693861 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:53:47.694191 (+ 330us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:53:47.694207 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 00:53:47.694278 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 00:53:47.694366 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260806 00:53:47.697429 30098 raft_consensus.cc:1261] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49 [term 1 FOLLOWER]: Refusing update from remote peer d1cd6e40982040bebe866c1a00d067ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:47.697427 29693 raft_consensus.cc:1261] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c [term 1 FOLLOWER]: Refusing update from remote peer d1cd6e40982040bebe866c1a00d067ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:47.697907 30200 consensus_queue.cc:1035] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:47.698081 30209 consensus_queue.cc:1035] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:47.699626 30195 tablet_replica.cc:439] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49: TxnStatusTablet state changed. Reason: New leader d1cd6e40982040bebe866c1a00d067ce. Latest consensus state: current_term: 1 leader_uuid: "d1cd6e40982040bebe866c1a00d067ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } } }
I20260806 00:53:47.699755 30195 tablet_replica.cc:442] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:53:47.701004 30194 tablet_replica.cc:439] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c: TxnStatusTablet state changed. Reason: New leader d1cd6e40982040bebe866c1a00d067ce. Latest consensus state: current_term: 1 leader_uuid: "d1cd6e40982040bebe866c1a00d067ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } } }
I20260806 00:53:47.701139 30194 tablet_replica.cc:442] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:53:47.703117 30209 tablet_replica.cc:439] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1cd6e40982040bebe866c1a00d067ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } } }
I20260806 00:53:47.703197 30204 mvcc.cc:205] Tried to move back new op lower bound from 7315364363043905536 to 7315364362952708096. Current Snapshot: MvccSnapshot[applied={T|T < 7315364363043905536}]
I20260806 00:53:47.703255 30209 tablet_replica.cc:442] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:53:47.703380 30209 tablet_replica.cc:439] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1cd6e40982040bebe866c1a00d067ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } } }
I20260806 00:53:47.703456 30209 tablet_replica.cc:442] T 67209770726b4cd5ac0f38091ecf685f P d1cd6e40982040bebe866c1a00d067ce: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:53:47.704041 30194 tablet_replica.cc:439] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1cd6e40982040bebe866c1a00d067ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } } }
I20260806 00:53:47.704154 30194 tablet_replica.cc:442] T 67209770726b4cd5ac0f38091ecf685f P ae4f2afdb7644fb3b21179127b0b956c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:53:47.705598 30195 tablet_replica.cc:439] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1cd6e40982040bebe866c1a00d067ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 } } peers { permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 } } peers { permanent_uuid: "eb6f8153541a4c1c990d2ebe763caa49" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41927 } } }
I20260806 00:53:47.705726 30195 tablet_replica.cc:442] T 67209770726b4cd5ac0f38091ecf685f P eb6f8153541a4c1c990d2ebe763caa49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:53:47.713722 30187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.709395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39890 (local address 127.27.77.195:41927)
0806 00:53:47.709694 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.709697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.709707 (+ 10us) server_negotiation.cc:407] Connection header received
0806 00:53:47.709772 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.709774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.709821 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.709900 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:47.710188 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.710693 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.712112 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.712385 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.712495 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.712780 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:47.713255 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:47.713353 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.713460 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.713514 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226}
I20260806 00:53:47.725063 30180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.721449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:55911 (local address 127.27.77.194:40357)
0806 00:53:47.721639 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.721643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.721739 (+ 96us) server_negotiation.cc:407] Connection header received
0806 00:53:47.721851 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.721854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.721895 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.721978 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:47.722262 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.722734 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.724016 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.724240 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.724322 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.724639 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:47.724704 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:47.724758 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.724883 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.724925 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260806 00:53:47.725095 30253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.721289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:40357 (local address 127.27.77.195:55911)
0806 00:53:47.721646 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:47.721662 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:47.721769 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:47.722009 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:47.722012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:47.722020 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:53:47.722149 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.722155 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.722835 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.722839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.723869 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.723881 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.724350 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.724353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.724452 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.724505 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:53:47.724785 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:53:47.724792 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 00:53:47.724857 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 00:53:47.724936 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":150,"threads_started":1}
I20260806 00:53:47.732530 29693 raft_consensus.cc:1261] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c [term 1 FOLLOWER]: Refusing update from remote peer eb6f8153541a4c1c990d2ebe763caa49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:47.732854 29886 raft_consensus.cc:1261] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce [term 1 FOLLOWER]: Refusing update from remote peer eb6f8153541a4c1c990d2ebe763caa49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:47.733460 30195 consensus_queue.cc:1035] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae4f2afdb7644fb3b21179127b0b956c" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:47.733860 30195 consensus_queue.cc:1035] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1cd6e40982040bebe866c1a00d067ce" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:47.758776 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.754395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:41927 (local address 127.27.77.194:58409)
0806 00:53:47.754772 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:47.754794 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:47.754915 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:47.755297 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:47.755301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:47.755310 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:53:47.755452 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.755458 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.756447 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.756451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.757449 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:47.757462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.758065 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.758069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:47.758155 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.758207 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:53:47.758520 (+ 313us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:53:47.758524 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 00:53:47.758573 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 00:53:47.758642 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":121,"threads_started":1}
I20260806 00:53:47.758778 30187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.754831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:58409 (local address 127.27.77.195:41927)
0806 00:53:47.755045 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.755050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.755069 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:53:47.755132 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.755134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.755186 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.755282 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:47.755625 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.756308 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.757596 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.757943 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.758063 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.758351 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:47.758439 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:47.758520 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.758585 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.758643 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:53:47.789023 29470 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:53:47.789216 29470 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:53:47.791731 29470 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 929cbad3c1b648f79f1ea260409f7d08: Sending DeleteTablet for 3 replicas of tablet ac7bc7e4c9754f5d921c89da8f371ff9
I20260806 00:53:47.792484 29666 tablet_service.cc:1514] Processing DeleteTablet for tablet ac7bc7e4c9754f5d921c89da8f371ff9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:47 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:43618
I20260806 00:53:47.792706 30217 tablet_replica.cc:330] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c: stopping tablet replica
I20260806 00:53:47.792841 30217 raft_consensus.cc:2229] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:53:47.793002 30217 raft_consensus.cc:2258] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:47.793534 30065 tablet_service.cc:1514] Processing DeleteTablet for tablet ac7bc7e4c9754f5d921c89da8f371ff9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:47 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:39886
I20260806 00:53:47.793730 30218 tablet_replica.cc:330] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49: stopping tablet replica
I20260806 00:53:47.793839 30218 raft_consensus.cc:2229] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:53:47.793851 30217 ts_tablet_manager.cc:1905] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:47.794173 30218 raft_consensus.cc:2258] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:47.795154 30218 ts_tablet_manager.cc:1905] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:47.795825 29859 tablet_service.cc:1514] Processing DeleteTablet for tablet ac7bc7e4c9754f5d921c89da8f371ff9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:47 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:60564
I20260806 00:53:47.796046 30216 tablet_replica.cc:330] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce: stopping tablet replica
I20260806 00:53:47.796170 30216 raft_consensus.cc:2229] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:53:47.796278 30216 raft_consensus.cc:2258] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:47.797248 30216 ts_tablet_manager.cc:1905] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:47.799921 30218 ts_tablet_manager.cc:1918] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:53:47.799998 30218 log.cc:1196] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/ac7bc7e4c9754f5d921c89da8f371ff9
I20260806 00:53:47.800056 30216 ts_tablet_manager.cc:1918] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:53:47.800130 30216 log.cc:1196] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/ac7bc7e4c9754f5d921c89da8f371ff9
I20260806 00:53:47.800285 30218 ts_tablet_manager.cc:1939] T ac7bc7e4c9754f5d921c89da8f371ff9 P eb6f8153541a4c1c990d2ebe763caa49: Deleting consensus metadata
I20260806 00:53:47.800385 30216 ts_tablet_manager.cc:1939] T ac7bc7e4c9754f5d921c89da8f371ff9 P d1cd6e40982040bebe866c1a00d067ce: Deleting consensus metadata
I20260806 00:53:47.801115 29458 catalog_manager.cc:4841] TS d1cd6e40982040bebe866c1a00d067ce (127.27.77.194:40357): tablet ac7bc7e4c9754f5d921c89da8f371ff9 (table test-table [id=33804d24076d4d0b84a4030523dc025f]) successfully deleted
I20260806 00:53:47.801301 29458 catalog_manager.cc:4841] TS eb6f8153541a4c1c990d2ebe763caa49 (127.27.77.195:41927): tablet ac7bc7e4c9754f5d921c89da8f371ff9 (table test-table [id=33804d24076d4d0b84a4030523dc025f]) successfully deleted
I20260806 00:53:47.802467 30217 ts_tablet_manager.cc:1918] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:53:47.802551 30217 log.cc:1196] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977619051187-27959-0/minicluster-data/ts-0/wal/wals/ac7bc7e4c9754f5d921c89da8f371ff9
I20260806 00:53:47.802820 30217 ts_tablet_manager.cc:1939] T ac7bc7e4c9754f5d921c89da8f371ff9 P ae4f2afdb7644fb3b21179127b0b956c: Deleting consensus metadata
I20260806 00:53:47.803653 29455 catalog_manager.cc:4841] TS ae4f2afdb7644fb3b21179127b0b956c (127.27.77.193:36265): tablet ac7bc7e4c9754f5d921c89da8f371ff9 (table test-table [id=33804d24076d4d0b84a4030523dc025f]) successfully deleted
Aug 06 00:53:47 dist-test-slave-sb2z krb5kdc[29409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977627, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.77.193@KRBTEST.COM
I20260806 00:53:47.817116 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.803984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43632 (local address 127.27.77.193:36265)
0806 00:53:47.804160 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.804165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.804426 (+ 261us) server_negotiation.cc:407] Connection header received
0806 00:53:47.804542 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.804545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.804599 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.804685 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:47.805586 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.806354 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.807638 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.807990 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.808111 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.812282 (+ 4171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:47.812300 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:47.812321 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:47.812353 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:47.814707 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:47.815318 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:47.815325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:47.815327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:47.815436 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:47.815824 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:47.815828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:47.815830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:47.816209 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:47.816449 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.816758 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.816901 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 00:53:47.817631 29744 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43632
I20260806 00:53:47.829929 29541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:47.819743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42450 (local address 127.27.77.254:43793)
0806 00:53:47.820084 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:47.820089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:47.820721 (+ 632us) server_negotiation.cc:407] Connection header received
0806 00:53:47.820850 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:47.820853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:47.820912 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:47.821011 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:47.821875 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.822706 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.823886 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:47.824247 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:47.824354 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:47.826080 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:47.826084 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:47.826087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:47.826120 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:47.827997 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:47.828509 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:47.828513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:47.828515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:47.828616 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:47.828943 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:47.828947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:47.828948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:47.829181 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:47.829314 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:47.829628 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 00:53:47.829756 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227}
W20260806 00:53:47.830461 29470 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42450
I20260806 00:53:47.832190 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 29572
I20260806 00:53:47.864918 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 29765
I20260806 00:53:47.877883 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 29973
I20260806 00:53:47.892468 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 29429
2026-08-06T00:53:47Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3125 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:47 dist-test-slave-sb2z krb5kdc[30268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:47 dist-test-slave-sb2z krb5kdc[30268](info): set up 2 sockets
Aug 06 00:53:47 dist-test-slave-sb2z krb5kdc[30268](info): commencing operation
krb5kdc: starting...
W20260806 00:53:49.107172 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:53:49 dist-test-slave-sb2z krb5kdc[30268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:53:49.242576 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:43717
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:43717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:49.359176 30288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:49.359516 30288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:49.359634 30288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:49.363715 30288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:49.363811 30288 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 00:53:49.363842 30288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:49.363895 30288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:49.363973 30288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:49.364013 30288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:49.369346 30288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:43717
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:43717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:49.370487 30288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:49.371598 30288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:49.380470 30297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:49.380541 30288 server_base.cc:1034] running on GCE node
W20260806 00:53:49.380481 30295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:49.380481 30301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:49.381407 30288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:49.382519 30288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:49.383708 30288 hybrid_clock.cc:648] HybridClock initialized: now 1785977629383688 us; error 32 us; skew 500 ppm
Aug 06 00:53:49 dist-test-slave-sb2z krb5kdc[30268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977629, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:49.386674 30288 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:53:49.387751 30288 webserver.cc:460] Webserver started at http://127.27.77.254:40769/ using document root <none> and password file <none>
I20260806 00:53:49.388409 30288 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:49.388525 30288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:49.388772 30288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:49.390933 30288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "9bc26f48adbd4a29b10ff375d41ebcf2"
format_stamp: "Formatted at 2026-08-06 00:53:49 on dist-test-slave-sb2z"
server_key: "217a6d54ade71855791364a616c3804b"
server_key_iv: "a0d59827d4667fd98f0b2f6b697775d7"
server_key_version: "encryptionkey@0"
I20260806 00:53:49.391620 30288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "9bc26f48adbd4a29b10ff375d41ebcf2"
format_stamp: "Formatted at 2026-08-06 00:53:49 on dist-test-slave-sb2z"
server_key: "217a6d54ade71855791364a616c3804b"
server_key_iv: "a0d59827d4667fd98f0b2f6b697775d7"
server_key_version: "encryptionkey@0"
I20260806 00:53:49.395644 30288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:53:49.398236 30311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:49.399663 30288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:53:49.399811 30288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "9bc26f48adbd4a29b10ff375d41ebcf2"
format_stamp: "Formatted at 2026-08-06 00:53:49 on dist-test-slave-sb2z"
server_key: "217a6d54ade71855791364a616c3804b"
server_key_iv: "a0d59827d4667fd98f0b2f6b697775d7"
server_key_version: "encryptionkey@0"
I20260806 00:53:49.399946 30288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:49.412388 30288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:49.415766 30288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:49.415995 30288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:49.424801 30288 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:43717
I20260806 00:53:49.424834 30418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:43717 every 8 connection(s)
I20260806 00:53:49.425788 30288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:49.429004 30419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:49.429723 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 30288
I20260806 00:53:49.429858 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:53:49.430189 27959 external_mini_cluster.cc:1428] Setting key 0b50477e87cd327f53394e8c3ce9aa61
I20260806 00:53:49.436584 30419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2: Bootstrap starting.
I20260806 00:53:49.439018 30419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2: Neither blocks nor log segments found. Creating new log.
Aug 06 00:53:49 dist-test-slave-sb2z krb5kdc[30268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:49.439903 30419 log.cc:824] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:49.442160 30419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2: No bootstrap required, opened a new log
I20260806 00:53:49.445480 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:49.431791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59324 (local address 127.27.77.254:43717)
0806 00:53:49.432288 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:49.432304 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:49.432341 (+ 37us) server_negotiation.cc:407] Connection header received
0806 00:53:49.433122 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:49.433142 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:49.433481 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:49.433840 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:49.434686 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.435743 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.436621 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.437175 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.437353 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:49.440050 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:49.440057 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:49.440084 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:49.440122 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:49.442393 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:49.442850 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:49.442863 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:49.442869 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:49.442991 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:49.443297 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:49.443300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:49.443302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:49.443789 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:49.444005 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:49.444294 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 00:53:49.444677 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":151,"threads_started":1}
I20260806 00:53:49.445403 30419 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc26f48adbd4a29b10ff375d41ebcf2" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43717 } }
I20260806 00:53:49.445727 30419 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:49.445812 30419 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc26f48adbd4a29b10ff375d41ebcf2, State: Initialized, Role: FOLLOWER
I20260806 00:53:49.446277 30419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc26f48adbd4a29b10ff375d41ebcf2" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43717 } }
I20260806 00:53:49.446439 30419 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:49.446529 30419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:49.446640 30419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:49.447917 30419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc26f48adbd4a29b10ff375d41ebcf2" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43717 } }
I20260806 00:53:49.448289 30419 leader_election.cc:302] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bc26f48adbd4a29b10ff375d41ebcf2; no voters:
I20260806 00:53:49.448588 30419 leader_election.cc:288] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:49.448735 30425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:49.449000 30425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [term 1 LEADER]: Becoming Leader. State: Replica: 9bc26f48adbd4a29b10ff375d41ebcf2, State: Running, Role: LEADER
I20260806 00:53:49.449360 30425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc26f48adbd4a29b10ff375d41ebcf2" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43717 } }
I20260806 00:53:49.450008 30419 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:49.451493 30426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bc26f48adbd4a29b10ff375d41ebcf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc26f48adbd4a29b10ff375d41ebcf2" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43717 } } }
I20260806 00:53:49.451562 30427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bc26f48adbd4a29b10ff375d41ebcf2. Latest consensus state: current_term: 1 leader_uuid: "9bc26f48adbd4a29b10ff375d41ebcf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc26f48adbd4a29b10ff375d41ebcf2" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43717 } } }
I20260806 00:53:49.451725 30426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:49.451771 30427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2 [sys.catalog]: This master's current role is: LEADER
W20260806 00:53:49.455598 30447 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:53:49.455683 30447 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:53:49.455765 30441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:49.456548 30441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:49.463464 30441 catalog_manager.cc:1347] Generated new cluster ID: 2ec0a133d3bd499ebc00db64d50c5614
I20260806 00:53:49.463594 30441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:49.488456 30441 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:49.489588 30441 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:49.499943 30441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2: Generated new TSK 0
I20260806 00:53:49.500765 30441 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:53:49.539438 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:43717
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:49.658761 30462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:49.659042 30462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:49.659090 30462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:49.663240 30462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:49.663304 30462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:49.663343 30462 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 00:53:49.663357 30462 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:53:49.663389 30462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:53:49.668654 30462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.77.254:43717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:49.669663 30462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:49.670643 30462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:49.680492 30471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:49.680486 30474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:49.680486 30470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:49.680671 30462 server_base.cc:1034] running on GCE node
I20260806 00:53:49.681217 30462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:49.681911 30462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:49.683132 30462 hybrid_clock.cc:648] HybridClock initialized: now 1785977629683097 us; error 44 us; skew 500 ppm
Aug 06 00:53:49 dist-test-slave-sb2z krb5kdc[30268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977629, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:49.686277 30462 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:53:49.687263 30462 webserver.cc:460] Webserver started at http://127.27.77.193:38939/ using document root <none> and password file <none>
I20260806 00:53:49.687939 30462 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:49.688046 30462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:49.688294 30462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:49.690554 30462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "755b7d50fe404f5e99a93cda6285c0e5"
format_stamp: "Formatted at 2026-08-06 00:53:49 on dist-test-slave-sb2z"
server_key: "94b27c26cc2d05bfc8987a0f98fc1c16"
server_key_iv: "8d9cabf231e59171cc7412fcdbe15af6"
server_key_version: "encryptionkey@0"
I20260806 00:53:49.691166 30462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "755b7d50fe404f5e99a93cda6285c0e5"
format_stamp: "Formatted at 2026-08-06 00:53:49 on dist-test-slave-sb2z"
server_key: "94b27c26cc2d05bfc8987a0f98fc1c16"
server_key_iv: "8d9cabf231e59171cc7412fcdbe15af6"
server_key_version: "encryptionkey@0"
I20260806 00:53:49.695079 30462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:53:49.697654 30484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:49.698997 30462 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:53:49.699152 30462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "755b7d50fe404f5e99a93cda6285c0e5"
format_stamp: "Formatted at 2026-08-06 00:53:49 on dist-test-slave-sb2z"
server_key: "94b27c26cc2d05bfc8987a0f98fc1c16"
server_key_iv: "8d9cabf231e59171cc7412fcdbe15af6"
server_key_version: "encryptionkey@0"
I20260806 00:53:49.699272 30462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:49.712575 30462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:49.715390 30462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:49.715674 30462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:49.717190 30462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:53:49.717263 30462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:49.717337 30462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:53:49.717381 30462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:53:49 dist-test-slave-sb2z krb5kdc[30268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977629, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:49.728808 30462 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:44301
I20260806 00:53:49.728930 30644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:44301 every 8 connection(s)
I20260806 00:53:49.730108 30462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:53:49.732301 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:49.718092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:57749 (local address 127.27.77.254:43717)
0806 00:53:49.718261 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:49.718266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:49.719116 (+ 850us) server_negotiation.cc:407] Connection header received
0806 00:53:49.720145 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:49.720148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:49.720205 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:49.720287 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:49.721649 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.722560 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.723947 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.724223 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.724340 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:49.728072 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:49.728075 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:49.728078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:49.728106 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:49.729884 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:49.730578 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:49.730582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:49.730584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:49.730683 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:49.731136 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:49.731143 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:49.731145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:49.731403 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:49.731493 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:49.732061 (+ 568us) server_negotiation.cc:299] Negotiation successful
0806 00:53:49.732174 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 00:53:49.733047 30497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:49.718452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43717 (local address 127.27.77.193:57749)
0806 00:53:49.718946 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:49.718996 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:49.719936 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:49.720613 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:49.720624 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:49.721032 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:49.721489 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:49.721503 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.722729 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.722735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:49.723813 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:49.723824 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.724399 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.724403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:49.724626 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:49.725269 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:49.725287 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:49.727840 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:49.730050 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:49.730055 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:49.730067 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:49.730402 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:49.730809 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:49.730813 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:49.730815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:49.730995 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:49.731503 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:49.731516 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:49.731806 (+ 290us) client_negotiation.cc:769] Sending connection context
0806 00:53:49.731960 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 00:53:49.732215 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":143,"threads_started":1}
I20260806 00:53:49.734411 30645 heartbeater.cc:347] Connected to a master server at 127.27.77.254:43717
I20260806 00:53:49.734648 30645 heartbeater.cc:464] Registering TS with master...
I20260806 00:53:49.735167 30645 heartbeater.cc:511] Master 127.27.77.254:43717 requested a full tablet report, sending...
I20260806 00:53:49.736883 30352 ts_manager.cc:194] Registered new tserver with Master: 755b7d50fe404f5e99a93cda6285c0e5 (127.27.77.193:44301)
I20260806 00:53:49.737090 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 30462
I20260806 00:53:49.737226 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:53:49.737578 27959 external_mini_cluster.cc:1428] Setting key be98560ce6072f95e2b25025b2d6363c
I20260806 00:53:49.738579 30352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:57749
I20260806 00:53:49.745923 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:49.737937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:55803 (local address 127.27.77.254:43717)
0806 00:53:49.738079 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:49.738084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:49.738097 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:53:49.738143 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:49.738146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:49.738193 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:49.738298 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:49.738799 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.739569 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.741017 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.741240 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.741367 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:49.742600 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:49.742604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:49.742606 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:49.742632 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:49.744158 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:49.744649 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:49.744653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:49.744654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:49.744739 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:49.745042 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:49.745045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:49.745047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:49.745232 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:49.745346 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:49.745633 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 00:53:49.745754 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:53:49.745930 30655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:49.736853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43717 (local address 127.27.77.193:55803)
0806 00:53:49.737250 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:49.737268 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:49.737362 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:49.738319 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:49.738322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:49.738534 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:49.738637 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:49.738643 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.739852 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.739858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:49.740851 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:49.740867 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.741376 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.741379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:49.741483 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:49.742071 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:49.742083 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:49.742432 (+ 349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:49.744280 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:49.744285 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:49.744287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:49.744541 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:49.744837 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:49.744842 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:49.744843 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:49.744938 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:49.745336 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:49.745339 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:49.745447 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 00:53:49.745567 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 00:53:49.745734 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:53:49.770646 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:43717
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:49.888056 30660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:49.888322 30660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:49.888371 30660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:49.892446 30660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:49.892516 30660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:49.892556 30660 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 00:53:49.892570 30660 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:53:49.892603 30660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:53:49.897994 30660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.77.254:43717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:49.899044 30660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:49.900041 30660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:49.910112 30669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:49.910112 30665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:49.910218 30666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:49.910499 30660 server_base.cc:1034] running on GCE node
I20260806 00:53:49.910899 30660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:49.911607 30660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:49.912793 30660 hybrid_clock.cc:648] HybridClock initialized: now 1785977629912771 us; error 29 us; skew 500 ppm
Aug 06 00:53:49 dist-test-slave-sb2z krb5kdc[30268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977629, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:49.916285 30660 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:53:49.917214 30660 webserver.cc:460] Webserver started at http://127.27.77.194:39393/ using document root <none> and password file <none>
I20260806 00:53:49.917815 30660 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:49.917881 30660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:49.918056 30660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:49.920209 30660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "bff4d7d0085e4c7f921fd3502c0374eb"
format_stamp: "Formatted at 2026-08-06 00:53:49 on dist-test-slave-sb2z"
server_key: "3c2801b5e902f6dbf0caf11d49195b57"
server_key_iv: "d16483f9445e4a34bf552e175f590612"
server_key_version: "encryptionkey@0"
I20260806 00:53:49.920792 30660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "bff4d7d0085e4c7f921fd3502c0374eb"
format_stamp: "Formatted at 2026-08-06 00:53:49 on dist-test-slave-sb2z"
server_key: "3c2801b5e902f6dbf0caf11d49195b57"
server_key_iv: "d16483f9445e4a34bf552e175f590612"
server_key_version: "encryptionkey@0"
I20260806 00:53:49.924592 30660 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:53:49.927080 30678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:49.928582 30660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:53:49.928715 30660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "bff4d7d0085e4c7f921fd3502c0374eb"
format_stamp: "Formatted at 2026-08-06 00:53:49 on dist-test-slave-sb2z"
server_key: "3c2801b5e902f6dbf0caf11d49195b57"
server_key_iv: "d16483f9445e4a34bf552e175f590612"
server_key_version: "encryptionkey@0"
I20260806 00:53:49.928828 30660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:49.954071 30660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:49.957222 30660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:49.957437 30660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:49.958994 30660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:53:49.959084 30660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:49.959177 30660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:53:49.959231 30660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:53:49 dist-test-slave-sb2z krb5kdc[30268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977629, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:49.970680 30660 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:38375
I20260806 00:53:49.970764 30834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:38375 every 8 connection(s)
I20260806 00:53:49.971882 30660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:53:49.974261 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:49.959996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:46025 (local address 127.27.77.254:43717)
0806 00:53:49.960137 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:49.960141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:49.961295 (+ 1154us) server_negotiation.cc:407] Connection header received
0806 00:53:49.962343 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:49.962347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:49.962416 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:49.962507 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:49.964128 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.965038 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.966992 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.967303 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.967436 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:49.970560 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:49.970563 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:49.970565 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:49.970592 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:49.972287 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:49.972919 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:49.972922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:49.972924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:49.973009 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:49.973304 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:49.973306 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:49.973308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:49.973485 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:49.973585 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:49.974036 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 00:53:49.974142 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:53:49.975111 30688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:49.960581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43717 (local address 127.27.77.194:46025)
0806 00:53:49.961134 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:49.961182 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:49.962079 (+ 897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:49.962993 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:49.963000 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:49.963336 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:49.963925 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:49.963947 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.965225 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.965229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:49.966835 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:49.966850 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.967477 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.967482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:49.967728 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:49.968354 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:49.968370 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:49.970283 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:49.972450 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:49.972456 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:49.972470 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:49.972806 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:49.973096 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:49.973098 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:49.973099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:49.973221 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:49.973601 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:49.973606 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:49.973826 (+ 220us) client_negotiation.cc:769] Sending connection context
0806 00:53:49.973980 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 00:53:49.974224 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":130,"threads_started":1}
I20260806 00:53:49.976441 30836 heartbeater.cc:347] Connected to a master server at 127.27.77.254:43717
I20260806 00:53:49.976662 30836 heartbeater.cc:464] Registering TS with master...
I20260806 00:53:49.977272 30836 heartbeater.cc:511] Master 127.27.77.254:43717 requested a full tablet report, sending...
I20260806 00:53:49.978420 30342 ts_manager.cc:194] Registered new tserver with Master: bff4d7d0085e4c7f921fd3502c0374eb (127.27.77.194:38375)
I20260806 00:53:49.978914 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 30660
I20260806 00:53:49.979041 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:53:49.979226 30342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:46025
I20260806 00:53:49.979306 27959 external_mini_cluster.cc:1428] Setting key 16022b9fc328dcf1dae0db376333717d
I20260806 00:53:49.987809 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:49.979029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:53209 (local address 127.27.77.254:43717)
0806 00:53:49.979497 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:49.979501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:49.979814 (+ 313us) server_negotiation.cc:407] Connection header received
0806 00:53:49.979935 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:49.979938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:49.979987 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:49.980078 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:49.980558 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.981302 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.982572 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.982869 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.983018 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:49.984226 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:49.984232 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:49.984237 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:49.984270 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:49.985888 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:49.986456 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:49.986459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:49.986460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:49.986539 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:49.986865 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:49.986867 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:49.986869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:49.987101 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:49.987180 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:49.987601 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 00:53:49.987696 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408}
I20260806 00:53:49.988039 30850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:49.978931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43717 (local address 127.27.77.194:53209)
0806 00:53:49.979711 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:49.979728 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:49.979833 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:49.980098 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:49.980101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:49.980304 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:49.980436 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:49.980446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.981450 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.981456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:49.982430 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:49.982445 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:49.983019 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:49.983023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:49.983114 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:49.983694 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:49.983706 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:49.984065 (+ 359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:49.986018 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:49.986022 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:49.986025 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:49.986344 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:49.986645 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:49.986649 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:49.986652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:49.986768 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:49.987214 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:49.987220 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:49.987354 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 00:53:49.987510 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 00:53:49.987743 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:53:50.011121 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:43717
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:50.129040 30853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:50.129290 30853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:50.129338 30853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:50.133335 30853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:50.133394 30853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:50.133436 30853 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 00:53:50.133450 30853 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:53:50.133482 30853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:53:50.138814 30853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.77.254:43717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:50.139827 30853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:50.140838 30853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:50.150893 30862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:50.150858 30861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:50.150897 30866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:50.151165 30853 server_base.cc:1034] running on GCE node
I20260806 00:53:50.151696 30853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:50.152432 30853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:50.153636 30853 hybrid_clock.cc:648] HybridClock initialized: now 1785977630153622 us; error 27 us; skew 500 ppm
Aug 06 00:53:50 dist-test-slave-sb2z krb5kdc[30268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977630, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:50.157107 30853 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:53:50.158047 30853 webserver.cc:460] Webserver started at http://127.27.77.195:43721/ using document root <none> and password file <none>
I20260806 00:53:50.158664 30853 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:50.158764 30853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:50.159027 30853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:50.161319 30853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "bc6e8671ca9b4737b7751a2db517b6bf"
format_stamp: "Formatted at 2026-08-06 00:53:50 on dist-test-slave-sb2z"
server_key: "d4fc6593fdaede084152a1d157fcb3bb"
server_key_iv: "c220e86ecd6bc27752859f02abd022fb"
server_key_version: "encryptionkey@0"
I20260806 00:53:50.161938 30853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "bc6e8671ca9b4737b7751a2db517b6bf"
format_stamp: "Formatted at 2026-08-06 00:53:50 on dist-test-slave-sb2z"
server_key: "d4fc6593fdaede084152a1d157fcb3bb"
server_key_iv: "c220e86ecd6bc27752859f02abd022fb"
server_key_version: "encryptionkey@0"
I20260806 00:53:50.165974 30853 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 00:53:50.168771 30874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:50.170203 30853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:53:50.170373 30853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "bc6e8671ca9b4737b7751a2db517b6bf"
format_stamp: "Formatted at 2026-08-06 00:53:50 on dist-test-slave-sb2z"
server_key: "d4fc6593fdaede084152a1d157fcb3bb"
server_key_iv: "c220e86ecd6bc27752859f02abd022fb"
server_key_version: "encryptionkey@0"
I20260806 00:53:50.170506 30853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:50.188581 30853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:50.192310 30853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:50.192543 30853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:50.194111 30853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:53:50.194193 30853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:50.194276 30853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:53:50.194332 30853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:53:50 dist-test-slave-sb2z krb5kdc[30268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977630, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:50.207406 30853 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:37977
I20260806 00:53:50.207487 31040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:37977 every 8 connection(s)
I20260806 00:53:50.208666 30853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:53:50.209089 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 30853
I20260806 00:53:50.209218 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:53:50.209542 27959 external_mini_cluster.cc:1428] Setting key fed64fb9d784f4226b788bfb7dd69991
I20260806 00:53:50.211198 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.195133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:35733 (local address 127.27.77.254:43717)
0806 00:53:50.195267 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:50.195271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:50.196527 (+ 1256us) server_negotiation.cc:407] Connection header received
0806 00:53:50.197643 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:50.197650 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:50.197718 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:50.197785 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:50.199731 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.200464 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.202007 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.202305 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.202429 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.206711 (+ 4282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:50.206715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:50.206717 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:50.206745 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:50.208539 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:50.209236 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:50.209240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:50.209242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:50.209353 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:50.209819 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:50.209823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:50.209825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:50.210078 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:50.210224 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:50.210832 (+ 608us) server_negotiation.cc:299] Negotiation successful
0806 00:53:50.210976 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:53:50.212177 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.195780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43717 (local address 127.27.77.195:35733)
0806 00:53:50.196349 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:50.196393 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:50.197345 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:50.198281 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:50.198290 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:50.198838 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:50.199506 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.199526 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.200621 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.200624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.201807 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.201818 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.202481 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.202486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.202673 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.203407 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:50.203436 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:50.206498 (+ 3062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:50.208677 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:50.208687 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:50.208704 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:50.209085 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:50.209485 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:50.209491 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:50.209494 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:50.209694 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:50.210221 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:50.210229 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:50.210544 (+ 315us) client_negotiation.cc:769] Sending connection context
0806 00:53:50.210762 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 00:53:50.211140 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":142,"threads_started":1}
I20260806 00:53:50.213392 31041 heartbeater.cc:347] Connected to a master server at 127.27.77.254:43717
I20260806 00:53:50.213579 31041 heartbeater.cc:464] Registering TS with master...
I20260806 00:53:50.214206 31041 heartbeater.cc:511] Master 127.27.77.254:43717 requested a full tablet report, sending...
I20260806 00:53:50.215268 30342 ts_manager.cc:194] Registered new tserver with Master: bc6e8671ca9b4737b7751a2db517b6bf (127.27.77.195:37977)
I20260806 00:53:50.216110 30342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:35733
I20260806 00:53:50.222937 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.215714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43717 (local address 127.27.77.195:33323)
0806 00:53:50.216021 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:50.216034 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:50.216134 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:50.216433 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:50.216436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:50.216680 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:50.216798 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.216804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.217553 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.217556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.218196 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.218206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.218648 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.218651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.218746 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.219216 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:50.219227 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:50.219592 (+ 365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:50.221200 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:50.221204 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:50.221206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:50.221403 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:50.221727 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:50.221746 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:50.221748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:50.221836 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:50.222361 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:50.222363 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:50.222438 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 00:53:50.222576 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 00:53:50.222742 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":100,"threads_started":1}
I20260806 00:53:50.223042 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.215784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:33323 (local address 127.27.77.254:43717)
0806 00:53:50.215920 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:50.215924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:50.216104 (+ 180us) server_negotiation.cc:407] Connection header received
0806 00:53:50.216247 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:50.216251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:50.216304 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:50.216401 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:50.216918 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.217416 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.218320 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.218536 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.218643 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.219732 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:50.219736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:50.219738 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:50.219768 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:50.221087 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:50.221513 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:50.221516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:50.221518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:50.221632 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:50.221932 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:50.221937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:50.221938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:50.222111 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:50.222225 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:50.222724 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 00:53:50.222868 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:53:50.223881 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 00:53:50 dist-test-slave-sb2z krb5kdc[30268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 00:53:50 dist-test-slave-sb2z krb5kdc[30268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:50.253037 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.243712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59326 (local address 127.27.77.254:43717)
0806 00:53:50.243856 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:50.243860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:50.243874 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:53:50.243918 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:50.243921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:50.243969 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:50.244048 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:50.244746 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.245307 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.246336 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.246611 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.246737 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.249554 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:50.249563 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:50.249565 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:50.249591 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:50.251238 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:50.251748 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:50.251753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:50.251755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:50.251843 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:50.252144 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:50.252148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:50.252149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:50.252338 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:50.252479 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:50.252736 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 00:53:50.252859 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:53:50.256094 30352 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:53:50.259063 30352 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:53:50.274932 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.267647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:44301 (local address 127.0.0.1:50254)
0806 00:53:50.268468 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:50.268520 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:50.268698 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:50.269416 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:50.269420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:50.269441 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:50.269542 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.269546 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.270918 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.270922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.272223 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.272235 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.274089 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.274095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.274330 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.274516 (+ 186us) client_negotiation.cc:769] Sending connection context
0806 00:53:50.274620 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 00:53:50.274747 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":83,"threads_started":1}
I20260806 00:53:50.275352 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.267793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50254 (local address 127.27.77.193:44301)
0806 00:53:50.268224 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:50.268231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:50.268646 (+ 415us) server_negotiation.cc:407] Connection header received
0806 00:53:50.268900 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:50.268907 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:50.269129 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:50.269255 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:50.269665 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.270786 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.272450 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.273821 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.274864 (+ 1043us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.275016 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:50.275088 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 00:53:50.275206 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":95,"threads_started":1}
I20260806 00:53:50.275763 31071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.268166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:37977 (local address 127.0.0.1:37018)
0806 00:53:50.268903 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:50.268918 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:50.269041 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:50.269638 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:50.269641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:50.269662 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:50.269792 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.269799 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.271291 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.271294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.272647 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.272659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.274545 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.274549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.274647 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.274668 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:53:50.274724 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 00:53:50.274773 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":96,"threads_started":1}
I20260806 00:53:50.276170 31070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.268379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:38375 (local address 127.0.0.1:59876)
0806 00:53:50.269271 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:50.269285 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:50.269366 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:50.270270 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:50.270274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:50.270295 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:50.270383 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.270389 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.271762 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.271766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.273019 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.273031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.275572 (+ 2541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.275579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.275753 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.275785 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 00:53:50.275871 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 00:53:50.275962 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":126,"threads_started":1}
I20260806 00:53:50.276822 31073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.268862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59876 (local address 127.27.77.194:38375)
0806 00:53:50.269840 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:50.269852 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:50.269873 (+ 21us) server_negotiation.cc:407] Connection header received
0806 00:53:50.269946 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:50.269951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:50.270117 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:50.270254 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:50.270519 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.271637 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.273167 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.275110 (+ 1943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.275245 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.275462 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:50.276507 (+ 1045us) server_negotiation.cc:299] Negotiation successful
0806 00:53:50.276626 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876,"thread_start_us":98,"threads_started":1}
I20260806 00:53:50.276973 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.268632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37018 (local address 127.27.77.195:37977)
0806 00:53:50.269143 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:50.269158 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:50.269183 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:53:50.269267 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:50.269272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:50.269490 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:50.269661 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:50.269941 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.271146 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.272809 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.273693 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.273809 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.276623 (+ 2814us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:50.276681 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 00:53:50.276781 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":134,"threads_started":1}
I20260806 00:53:50.278899 30549 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef79abc61f7467db46f44bb6415738b (DEFAULT_TABLE table=test-table [id=32714b9aa4334c38a153f7a0769f7ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:50.280198 30549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ef79abc61f7467db46f44bb6415738b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:50.280602 30945 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef79abc61f7467db46f44bb6415738b (DEFAULT_TABLE table=test-table [id=32714b9aa4334c38a153f7a0769f7ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:50.282676 30945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ef79abc61f7467db46f44bb6415738b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:50.287675 31077 tablet_bootstrap.cc:492] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5: Bootstrap starting.
I20260806 00:53:50.290083 31077 tablet_bootstrap.cc:654] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:50.290817 31078 tablet_bootstrap.cc:492] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf: Bootstrap starting.
I20260806 00:53:50.291055 31077 log.cc:824] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:50.290638 30750 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef79abc61f7467db46f44bb6415738b (DEFAULT_TABLE table=test-table [id=32714b9aa4334c38a153f7a0769f7ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:50.291929 30750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ef79abc61f7467db46f44bb6415738b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:50.293219 31078 tablet_bootstrap.cc:654] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:50.293556 31077 tablet_bootstrap.cc:492] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5: No bootstrap required, opened a new log
I20260806 00:53:50.293781 31077 ts_tablet_manager.cc:1397] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:53:50.294039 31078 log.cc:824] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:50.295746 31078 tablet_bootstrap.cc:492] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf: No bootstrap required, opened a new log
I20260806 00:53:50.295948 31078 ts_tablet_manager.cc:1397] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:53:50.298803 31078 raft_consensus.cc:348] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.299045 31078 raft_consensus.cc:374] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:50.299098 31078 raft_consensus.cc:729] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6e8671ca9b4737b7751a2db517b6bf, State: Initialized, Role: FOLLOWER
I20260806 00:53:50.299105 31077 raft_consensus.cc:348] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.299378 31077 raft_consensus.cc:374] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:50.299460 31077 raft_consensus.cc:729] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755b7d50fe404f5e99a93cda6285c0e5, State: Initialized, Role: FOLLOWER
I20260806 00:53:50.299599 31078 consensus_queue.cc:260] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.299932 31077 consensus_queue.cc:260] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.300591 31041 heartbeater.cc:503] Master 127.27.77.254:43717 was elected leader, sending a full tablet report...
I20260806 00:53:50.300659 31078 ts_tablet_manager.cc:1428] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:53:50.300907 30645 heartbeater.cc:503] Master 127.27.77.254:43717 was elected leader, sending a full tablet report...
I20260806 00:53:50.300940 31077 ts_tablet_manager.cc:1428] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5: Time spent starting tablet: real 0.007s user 0.002s sys 0.006s
I20260806 00:53:50.302470 31084 tablet_bootstrap.cc:492] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb: Bootstrap starting.
I20260806 00:53:50.305049 31084 tablet_bootstrap.cc:654] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:50.305994 31084 log.cc:824] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:50.307863 31084 tablet_bootstrap.cc:492] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb: No bootstrap required, opened a new log
I20260806 00:53:50.308064 31084 ts_tablet_manager.cc:1397] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260806 00:53:50.311148 31084 raft_consensus.cc:348] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.311360 31084 raft_consensus.cc:374] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:50.311393 31084 raft_consensus.cc:729] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff4d7d0085e4c7f921fd3502c0374eb, State: Initialized, Role: FOLLOWER
I20260806 00:53:50.311786 31084 consensus_queue.cc:260] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.312426 30836 heartbeater.cc:503] Master 127.27.77.254:43717 was elected leader, sending a full tablet report...
I20260806 00:53:50.312680 31084 ts_tablet_manager.cc:1428] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260806 00:53:50.422227 31081 raft_consensus.cc:482] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:53:50.422479 31081 raft_consensus.cc:504] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.423993 31081 leader_election.cc:288] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 755b7d50fe404f5e99a93cda6285c0e5 (127.27.77.193:44301), bff4d7d0085e4c7f921fd3502c0374eb (127.27.77.194:38375)
I20260806 00:53:50.428557 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.424270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:44301 (local address 127.27.77.195:60485)
0806 00:53:50.424438 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:50.424474 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:50.424583 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:50.424993 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:50.424996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:50.425021 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:50.425193 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.425201 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.426041 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.426047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.427285 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.427295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.428042 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.428046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.428208 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.428237 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 00:53:50.428330 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 00:53:50.428410 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":24}
I20260806 00:53:50.428548 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.424356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:60485 (local address 127.27.77.193:44301)
0806 00:53:50.424506 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:50.424513 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:50.424531 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:53:50.424730 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:50.424737 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:50.424795 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:50.424863 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:50.425355 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.425924 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.427415 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.427914 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.428064 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.428137 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:50.428347 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 00:53:50.428420 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:53:50.429195 30575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef79abc61f7467db46f44bb6415738b" candidate_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "755b7d50fe404f5e99a93cda6285c0e5" is_pre_election: true
I20260806 00:53:50.429539 30575 raft_consensus.cc:2454] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc6e8671ca9b4737b7751a2db517b6bf in term 0.
I20260806 00:53:50.429955 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.424602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:38375 (local address 127.27.77.195:54823)
0806 00:53:50.425073 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:50.425087 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:50.425195 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:50.425470 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:50.425473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:50.425490 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:50.426153 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.426164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.427195 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.427198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.428514 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.428526 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.429534 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.429537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.429636 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.429656 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:53:50.429743 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 00:53:50.429808 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":52,"thread_start_us":111,"threads_started":1}
I20260806 00:53:50.430014 31073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.424825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:54823 (local address 127.27.77.194:38375)
0806 00:53:50.424962 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:50.424967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:50.425182 (+ 215us) server_negotiation.cc:407] Connection header received
0806 00:53:50.425303 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:50.425307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:50.425361 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:50.425438 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:50.426300 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.427068 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.428674 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.429266 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.429364 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.429411 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:50.429780 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 00:53:50.429887 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:53:50.430079 30881 leader_election.cc:302] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755b7d50fe404f5e99a93cda6285c0e5, bc6e8671ca9b4737b7751a2db517b6bf; no voters:
I20260806 00:53:50.430390 31081 raft_consensus.cc:2781] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:53:50.430490 31081 raft_consensus.cc:482] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:53:50.430541 31081 raft_consensus.cc:3037] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:50.430837 30775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef79abc61f7467db46f44bb6415738b" candidate_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" is_pre_election: true
I20260806 00:53:50.431128 30775 raft_consensus.cc:2454] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc6e8671ca9b4737b7751a2db517b6bf in term 0.
I20260806 00:53:50.431954 31081 raft_consensus.cc:504] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.432297 31081 leader_election.cc:288] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [CANDIDATE]: Term 1 election: Requested vote from peers 755b7d50fe404f5e99a93cda6285c0e5 (127.27.77.193:44301), bff4d7d0085e4c7f921fd3502c0374eb (127.27.77.194:38375)
I20260806 00:53:50.432789 30775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef79abc61f7467db46f44bb6415738b" candidate_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff4d7d0085e4c7f921fd3502c0374eb"
I20260806 00:53:50.432839 30575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef79abc61f7467db46f44bb6415738b" candidate_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "755b7d50fe404f5e99a93cda6285c0e5"
I20260806 00:53:50.432917 30775 raft_consensus.cc:3037] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:50.432957 30575 raft_consensus.cc:3037] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:50.434342 30575 raft_consensus.cc:2454] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc6e8671ca9b4737b7751a2db517b6bf in term 1.
I20260806 00:53:50.434342 30775 raft_consensus.cc:2454] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc6e8671ca9b4737b7751a2db517b6bf in term 1.
I20260806 00:53:50.434688 30881 leader_election.cc:302] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755b7d50fe404f5e99a93cda6285c0e5, bc6e8671ca9b4737b7751a2db517b6bf; no voters:
I20260806 00:53:50.435057 31081 raft_consensus.cc:2781] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:50.435286 31081 raft_consensus.cc:686] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [term 1 LEADER]: Becoming Leader. State: Replica: bc6e8671ca9b4737b7751a2db517b6bf, State: Running, Role: LEADER
I20260806 00:53:50.435691 31081 consensus_queue.cc:237] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.438794 30352 catalog_manager.cc:5495] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf reported cstate change: term changed from 0 to 1, leader changed from <none> to bc6e8671ca9b4737b7751a2db517b6bf (127.27.77.195). New cstate: current_term: 1 leader_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } health_report { overall_health: HEALTHY } } }
W20260806 00:53:50.460266 31042 tablet.cc:2369] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:53:50.473189 30839 tablet.cc:2369] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:53:50.481622 30649 tablet.cc:2369] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 00:53:50 dist-test-slave-sb2z krb5kdc[30268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977629, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:50.494453 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.485303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.27.77.254:43717)
0806 00:53:50.485475 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:50.485481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:50.485655 (+ 174us) server_negotiation.cc:407] Connection header received
0806 00:53:50.485821 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:50.485824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:50.485868 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:50.485958 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:50.486631 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.487165 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.488052 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.488265 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.488380 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.491002 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:50.491005 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:50.491007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:50.491031 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:50.492528 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:50.493080 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:50.493083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:50.493084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:50.493162 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:50.493474 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:50.493477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:50.493478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:50.493673 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:50.493753 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:50.494188 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 00:53:50.494294 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 00:53:50.494515 31101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.485261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43717 (local address 127.0.0.1:59330)
0806 00:53:50.485561 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:50.485573 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:50.485721 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:50.485976 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:50.485978 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:50.486413 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:50.486506 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.486513 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.487293 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.487297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.487924 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.487934 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.488383 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.488386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.488477 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.489178 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:50.489210 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:50.490832 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:50.492669 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:50.492679 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:50.492684 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:50.492956 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:50.493281 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:50.493284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:50.493286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:50.493377 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:50.493797 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:50.493804 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:50.493917 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 00:53:50.494106 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 00:53:50.494239 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":84,"threads_started":1}
I20260806 00:53:50.498693 30352 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:59330:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 00:53:50.499497 30352 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:53:50.505106 30750 tablet_service.cc:1467] Processing CreateTablet for tablet 39ed74f274ae482ab0ec7dedc2118542 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f836fcf23de84be38bd8e173076ae28a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:53:50.505080 30549 tablet_service.cc:1467] Processing CreateTablet for tablet 39ed74f274ae482ab0ec7dedc2118542 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f836fcf23de84be38bd8e173076ae28a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:53:50.505364 30945 tablet_service.cc:1467] Processing CreateTablet for tablet 39ed74f274ae482ab0ec7dedc2118542 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f836fcf23de84be38bd8e173076ae28a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:53:50.505889 30549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39ed74f274ae482ab0ec7dedc2118542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:50.505889 30750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39ed74f274ae482ab0ec7dedc2118542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:50.506008 30945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39ed74f274ae482ab0ec7dedc2118542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:50.508844 31078 tablet_bootstrap.cc:492] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf: Bootstrap starting.
I20260806 00:53:50.508939 31084 tablet_bootstrap.cc:492] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb: Bootstrap starting.
I20260806 00:53:50.508939 31077 tablet_bootstrap.cc:492] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5: Bootstrap starting.
I20260806 00:53:50.509889 31078 tablet_bootstrap.cc:654] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:50.509904 31084 tablet_bootstrap.cc:654] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:50.509898 31077 tablet_bootstrap.cc:654] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:50.511289 31078 tablet_bootstrap.cc:492] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf: No bootstrap required, opened a new log
I20260806 00:53:50.511407 31078 ts_tablet_manager.cc:1397] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:53:50.511518 31084 tablet_bootstrap.cc:492] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb: No bootstrap required, opened a new log
I20260806 00:53:50.511643 31084 ts_tablet_manager.cc:1397] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 00:53:50.511801 31077 tablet_bootstrap.cc:492] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5: No bootstrap required, opened a new log
I20260806 00:53:50.511888 31077 ts_tablet_manager.cc:1397] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 00:53:50.511989 31078 raft_consensus.cc:348] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.512131 31078 raft_consensus.cc:374] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:50.512166 31078 raft_consensus.cc:729] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6e8671ca9b4737b7751a2db517b6bf, State: Initialized, Role: FOLLOWER
I20260806 00:53:50.512167 31084 raft_consensus.cc:348] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.512270 31084 raft_consensus.cc:374] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:50.512303 31084 raft_consensus.cc:729] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff4d7d0085e4c7f921fd3502c0374eb, State: Initialized, Role: FOLLOWER
I20260806 00:53:50.512295 31078 consensus_queue.cc:260] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.512352 31077 raft_consensus.cc:348] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.512472 31077 raft_consensus.cc:374] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:50.512527 31077 raft_consensus.cc:729] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755b7d50fe404f5e99a93cda6285c0e5, State: Initialized, Role: FOLLOWER
I20260806 00:53:50.512472 31084 consensus_queue.cc:260] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.512585 31078 ts_tablet_manager.cc:1428] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 00:53:50.512653 31077 consensus_queue.cc:260] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.512850 31081 tablet_replica.cc:439] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } } }
I20260806 00:53:50.512955 31077 ts_tablet_manager.cc:1428] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:53:50.513005 31081 tablet_replica.cc:442] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:53:50.513165 31087 tablet_replica.cc:439] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } } }
I20260806 00:53:50.513301 31087 tablet_replica.cc:442] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:53:50.514256 31083 tablet_replica.cc:439] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } } }
I20260806 00:53:50.514411 31083 tablet_replica.cc:442] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:53:50.514667 31084 ts_tablet_manager.cc:1428] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 00:53:50.641932 31083 raft_consensus.cc:482] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:53:50.642149 31083 raft_consensus.cc:504] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.643584 31083 leader_election.cc:288] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bff4d7d0085e4c7f921fd3502c0374eb (127.27.77.194:38375), bc6e8671ca9b4737b7751a2db517b6bf (127.27.77.195:37977)
I20260806 00:53:50.648289 31073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.644499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:54891 (local address 127.27.77.194:38375)
0806 00:53:50.644637 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:50.644640 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:50.644652 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:53:50.644701 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:50.644727 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:50.644785 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:50.644880 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:50.645188 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.645717 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.647099 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.647635 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.647758 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.647811 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:50.648057 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 00:53:50.648151 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 00:53:50.648326 31109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.643851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:38375 (local address 127.27.77.193:54891)
0806 00:53:50.644221 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:50.644236 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:50.644353 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:50.644880 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:50.644881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:50.644891 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:50.645066 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.645073 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.645825 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.645827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.646968 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.646979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.647804 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.647808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.647903 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.647931 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 00:53:50.648021 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 00:53:50.648083 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":25,"thread_start_us":157,"threads_started":1}
I20260806 00:53:50.648754 31107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.643854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:37977 (local address 127.27.77.193:43721)
0806 00:53:50.644626 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:50.644647 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:50.644749 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:50.644994 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:50.644996 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:50.645015 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:50.645607 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.645615 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.646389 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.646393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.647363 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.647372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.648236 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.648239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.648375 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.648413 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 00:53:50.648567 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 00:53:50.648628 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":23,"thread_start_us":130,"threads_started":1}
I20260806 00:53:50.648795 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.644338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:43721 (local address 127.27.77.195:37977)
0806 00:53:50.644468 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:50.644471 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:50.644742 (+ 271us) server_negotiation.cc:407] Connection header received
0806 00:53:50.644828 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:50.644834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:50.644886 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:50.644956 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:50.645728 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.646265 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.647478 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.648088 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.648207 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.648253 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:50.648581 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 00:53:50.648664 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:53:50.649143 30775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39ed74f274ae482ab0ec7dedc2118542" candidate_uuid: "755b7d50fe404f5e99a93cda6285c0e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" is_pre_election: true
I20260806 00:53:50.649363 30775 raft_consensus.cc:2454] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 755b7d50fe404f5e99a93cda6285c0e5 in term 0.
I20260806 00:53:50.649506 30971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39ed74f274ae482ab0ec7dedc2118542" candidate_uuid: "755b7d50fe404f5e99a93cda6285c0e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" is_pre_election: true
I20260806 00:53:50.649765 30971 raft_consensus.cc:2454] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 755b7d50fe404f5e99a93cda6285c0e5 in term 0.
I20260806 00:53:50.649864 30489 leader_election.cc:302] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755b7d50fe404f5e99a93cda6285c0e5, bff4d7d0085e4c7f921fd3502c0374eb; no voters:
I20260806 00:53:50.650216 31083 raft_consensus.cc:2781] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:53:50.650295 31083 raft_consensus.cc:482] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:53:50.650336 31083 raft_consensus.cc:3037] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:50.651610 31083 raft_consensus.cc:504] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.652017 31083 leader_election.cc:288] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [CANDIDATE]: Term 1 election: Requested vote from peers bff4d7d0085e4c7f921fd3502c0374eb (127.27.77.194:38375), bc6e8671ca9b4737b7751a2db517b6bf (127.27.77.195:37977)
I20260806 00:53:50.652395 30775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39ed74f274ae482ab0ec7dedc2118542" candidate_uuid: "755b7d50fe404f5e99a93cda6285c0e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff4d7d0085e4c7f921fd3502c0374eb"
I20260806 00:53:50.652465 30971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39ed74f274ae482ab0ec7dedc2118542" candidate_uuid: "755b7d50fe404f5e99a93cda6285c0e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6e8671ca9b4737b7751a2db517b6bf"
I20260806 00:53:50.652573 30775 raft_consensus.cc:3037] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:50.652587 30971 raft_consensus.cc:3037] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:50.653601 30971 raft_consensus.cc:2454] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 755b7d50fe404f5e99a93cda6285c0e5 in term 1.
I20260806 00:53:50.653721 30775 raft_consensus.cc:2454] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 755b7d50fe404f5e99a93cda6285c0e5 in term 1.
I20260806 00:53:50.653898 30487 leader_election.cc:302] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755b7d50fe404f5e99a93cda6285c0e5, bc6e8671ca9b4737b7751a2db517b6bf; no voters:
I20260806 00:53:50.654088 31083 raft_consensus.cc:2781] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:50.654321 31083 raft_consensus.cc:686] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 LEADER]: Becoming Leader. State: Replica: 755b7d50fe404f5e99a93cda6285c0e5, State: Running, Role: LEADER
I20260806 00:53:50.654683 31083 consensus_queue.cc:237] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.656989 31110 tablet_replica.cc:439] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5: TxnStatusTablet state changed. Reason: New leader 755b7d50fe404f5e99a93cda6285c0e5. Latest consensus state: current_term: 1 leader_uuid: "755b7d50fe404f5e99a93cda6285c0e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } } }
I20260806 00:53:50.657120 31110 tablet_replica.cc:442] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:53:50.657430 31113 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 00:53:50.657655 31113 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 00:53:50.658406 30354 catalog_manager.cc:5495] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 755b7d50fe404f5e99a93cda6285c0e5 (127.27.77.193). New cstate: current_term: 1 leader_uuid: "755b7d50fe404f5e99a93cda6285c0e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } health_report { overall_health: UNKNOWN } } }
I20260806 00:53:50.674440 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.670299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50270 (local address 127.27.77.193:44301)
0806 00:53:50.670459 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:50.670464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:50.670480 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:53:50.670559 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:50.670562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:50.670628 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:50.670739 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:50.671005 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.671816 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.672641 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.672997 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.673104 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.673395 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:50.673933 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:50.674066 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:50.674224 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 00:53:50.674293 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 00:53:50.674448 31101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.669951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:44301 (local address 127.0.0.1:50270)
0806 00:53:50.670163 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:50.670183 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:50.670298 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:50.670769 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:50.670771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:50.670778 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:53:50.670876 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.670880 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.671924 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.671926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.672523 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.672531 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.673125 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.673127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.673197 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.673246 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:53:50.674090 (+ 844us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:53:50.674109 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 00:53:50.674187 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 00:53:50.674264 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 00:53:50.680753 30775 raft_consensus.cc:1261] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Refusing update from remote peer 755b7d50fe404f5e99a93cda6285c0e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:50.680836 30971 raft_consensus.cc:1261] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Refusing update from remote peer 755b7d50fe404f5e99a93cda6285c0e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:50.681495 31110 consensus_queue.cc:1035] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:50.681686 31083 consensus_queue.cc:1035] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:50.685151 31081 tablet_replica.cc:439] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf: TxnStatusTablet state changed. Reason: New leader 755b7d50fe404f5e99a93cda6285c0e5. Latest consensus state: current_term: 1 leader_uuid: "755b7d50fe404f5e99a93cda6285c0e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } } }
I20260806 00:53:50.685290 31081 tablet_replica.cc:442] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:53:50.685567 31087 tablet_replica.cc:439] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb: TxnStatusTablet state changed. Reason: New leader 755b7d50fe404f5e99a93cda6285c0e5. Latest consensus state: current_term: 1 leader_uuid: "755b7d50fe404f5e99a93cda6285c0e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } } }
I20260806 00:53:50.685683 31087 tablet_replica.cc:442] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:53:50.689010 31083 tablet_replica.cc:439] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "755b7d50fe404f5e99a93cda6285c0e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } } }
I20260806 00:53:50.689152 31083 tablet_replica.cc:442] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:53:50.689276 31083 tablet_replica.cc:439] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "755b7d50fe404f5e99a93cda6285c0e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } } }
I20260806 00:53:50.689358 31083 tablet_replica.cc:442] T 39ed74f274ae482ab0ec7dedc2118542 P 755b7d50fe404f5e99a93cda6285c0e5: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:53:50.689455 31117 mvcc.cc:205] Tried to move back new op lower bound from 7315364375261011968 to 7315364375164289024. Current Snapshot: MvccSnapshot[applied={T|T < 7315364375261011968}]
I20260806 00:53:50.690375 31081 tablet_replica.cc:439] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "755b7d50fe404f5e99a93cda6285c0e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } } }
I20260806 00:53:50.690498 31081 tablet_replica.cc:442] T 39ed74f274ae482ab0ec7dedc2118542 P bc6e8671ca9b4737b7751a2db517b6bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:53:50.690891 31087 tablet_replica.cc:439] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "755b7d50fe404f5e99a93cda6285c0e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } } }
I20260806 00:53:50.691018 31087 tablet_replica.cc:442] T 39ed74f274ae482ab0ec7dedc2118542 P bff4d7d0085e4c7f921fd3502c0374eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:53:50.691160 31118 mvcc.cc:205] Tried to move back new op lower bound from 7315364375261011968 to 7315364375164289024. Current Snapshot: MvccSnapshot[applied={T|T < 7315364375261011968}]
I20260806 00:53:50.692441 31119 mvcc.cc:205] Tried to move back new op lower bound from 7315364375261011968 to 7315364375164289024. Current Snapshot: MvccSnapshot[applied={T|T < 7315364375261011968}]
I20260806 00:53:50.706251 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.701786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37022 (local address 127.27.77.195:37977)
0806 00:53:50.701902 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:50.701906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:50.701919 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:53:50.702142 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:50.702144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:50.702203 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:50.702288 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:50.702616 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.703225 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.704542 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.704851 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.704956 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.705259 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:50.705745 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:50.705837 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:50.705980 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 00:53:50.706040 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 00:53:50.718945 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.714872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:44301 (local address 127.27.77.195:48743)
0806 00:53:50.715060 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:50.715074 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:50.715188 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:50.715571 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:50.715577 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:50.715594 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:53:50.715720 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.715726 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.716766 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.716770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.717456 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.717473 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.718081 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.718085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.718172 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.718225 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:53:50.718653 (+ 428us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:53:50.718660 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 00:53:50.718727 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 00:53:50.718815 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 00:53:50.718959 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.715078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:48743 (local address 127.27.77.193:44301)
0806 00:53:50.715271 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:50.715275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:50.715289 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:53:50.715341 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:50.715344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:50.715394 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:50.715467 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:50.715841 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.716636 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.717576 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.717928 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.718041 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.718455 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:50.718553 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:50.718630 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:50.718765 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 00:53:50.718823 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:53:50.727772 30775 raft_consensus.cc:1261] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Refusing update from remote peer bc6e8671ca9b4737b7751a2db517b6bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:50.727911 30575 raft_consensus.cc:1261] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Refusing update from remote peer bc6e8671ca9b4737b7751a2db517b6bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:50.728380 31090 consensus_queue.cc:1035] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:50.728869 31081 consensus_queue.cc:1035] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:50.761924 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.757755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:58049 (local address 127.27.77.195:37977)
0806 00:53:50.757888 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:50.757893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:50.758197 (+ 304us) server_negotiation.cc:407] Connection header received
0806 00:53:50.758431 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:50.758435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:50.758497 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:50.758596 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:50.758925 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.759577 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.760550 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.760828 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.760969 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.761343 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:50.761441 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:50.761496 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:50.761734 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 00:53:50.761799 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:53:50.762411 31137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:50.757697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:37977 (local address 127.27.77.193:58049)
0806 00:53:50.758085 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:50.758099 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:50.758299 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:50.758613 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:50.758617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:50.758626 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:53:50.758798 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.758808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.759707 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.759710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.760414 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:50.760423 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:50.760971 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:50.760976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:50.761088 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:50.761162 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:53:50.761606 (+ 444us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:53:50.761614 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 00:53:50.762182 (+ 568us) client_negotiation.cc:240] Negotiation successful
0806 00:53:50.762261 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":104,"threads_started":1}
I20260806 00:53:50.794536 30342 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:53:50.794759 30342 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:53:50.797750 30342 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2: Sending DeleteTablet for 3 replicas of tablet 7ef79abc61f7467db46f44bb6415738b
I20260806 00:53:50.798825 30750 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ef79abc61f7467db46f44bb6415738b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:50 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:59876
I20260806 00:53:50.798942 30945 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ef79abc61f7467db46f44bb6415738b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:50 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:37018
I20260806 00:53:50.799510 30549 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ef79abc61f7467db46f44bb6415738b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:50 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:50254
I20260806 00:53:50.800032 31147 tablet_replica.cc:330] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5: stopping tablet replica
I20260806 00:53:50.800415 31147 raft_consensus.cc:2229] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:53:50.800675 31147 raft_consensus.cc:2258] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:50.800673 30342 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:53:50.801136 30342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:53:50.802498 31147 ts_tablet_manager.cc:1905] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:50.806571 31147 ts_tablet_manager.cc:1918] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:53:50.806787 31147 log.cc:1196] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/wal/wals/7ef79abc61f7467db46f44bb6415738b
I20260806 00:53:50.807235 31147 ts_tablet_manager.cc:1939] T 7ef79abc61f7467db46f44bb6415738b P 755b7d50fe404f5e99a93cda6285c0e5: Deleting consensus metadata
I20260806 00:53:50.807680 30945 tablet_service.cc:1467] Processing CreateTablet for tablet e786dfd3aae04ffcb270222627ca67c6 (DEFAULT_TABLE table=test-table [id=2bef6463fe9e48ad8cabc23830b8f5d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:50.808003 30945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e786dfd3aae04ffcb270222627ca67c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:50.810676 31078 tablet_bootstrap.cc:492] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf: Bootstrap starting.
I20260806 00:53:50.811203 30329 catalog_manager.cc:4841] TS 755b7d50fe404f5e99a93cda6285c0e5 (127.27.77.193:44301): tablet 7ef79abc61f7467db46f44bb6415738b (table test-table [id=32714b9aa4334c38a153f7a0769f7ef2]) successfully deleted
I20260806 00:53:50.811717 31078 tablet_bootstrap.cc:654] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:50.811933 30750 tablet_service.cc:1467] Processing CreateTablet for tablet e786dfd3aae04ffcb270222627ca67c6 (DEFAULT_TABLE table=test-table [id=2bef6463fe9e48ad8cabc23830b8f5d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:50.812225 30750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e786dfd3aae04ffcb270222627ca67c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:50.807709 31146 tablet_replica.cc:330] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf: stopping tablet replica
I20260806 00:53:50.812796 30549 tablet_service.cc:1467] Processing CreateTablet for tablet e786dfd3aae04ffcb270222627ca67c6 (DEFAULT_TABLE table=test-table [id=2bef6463fe9e48ad8cabc23830b8f5d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:50.813076 30549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e786dfd3aae04ffcb270222627ca67c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:50.813091 31146 raft_consensus.cc:2229] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:53:50.813474 31146 raft_consensus.cc:2258] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:50.814872 31084 tablet_bootstrap.cc:492] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb: Bootstrap starting.
I20260806 00:53:50.814963 31146 ts_tablet_manager.cc:1905] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:50.815474 31077 tablet_bootstrap.cc:492] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5: Bootstrap starting.
I20260806 00:53:50.815888 31084 tablet_bootstrap.cc:654] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:50.816550 31077 tablet_bootstrap.cc:654] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:50.817898 31144 tablet_replica.cc:330] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb: stopping tablet replica
I20260806 00:53:50.818158 31146 ts_tablet_manager.cc:1918] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:53:50.818216 31084 tablet_bootstrap.cc:492] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb: No bootstrap required, opened a new log
I20260806 00:53:50.818248 31146 log.cc:1196] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/7ef79abc61f7467db46f44bb6415738b
I20260806 00:53:50.818267 31144 raft_consensus.cc:2229] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:53:50.818295 31084 ts_tablet_manager.cc:1397] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:53:50.818617 31146 ts_tablet_manager.cc:1939] T 7ef79abc61f7467db46f44bb6415738b P bc6e8671ca9b4737b7751a2db517b6bf: Deleting consensus metadata
I20260806 00:53:50.818938 31084 raft_consensus.cc:348] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.819078 31084 raft_consensus.cc:374] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:50.819113 31084 raft_consensus.cc:729] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff4d7d0085e4c7f921fd3502c0374eb, State: Initialized, Role: FOLLOWER
I20260806 00:53:50.819211 31077 tablet_bootstrap.cc:492] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5: No bootstrap required, opened a new log
I20260806 00:53:50.819232 31084 consensus_queue.cc:260] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.819288 31077 ts_tablet_manager.cc:1397] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 00:53:50.819480 31084 ts_tablet_manager.cc:1428] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:53:50.819809 31077 raft_consensus.cc:348] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.819931 31077 raft_consensus.cc:374] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:50.819883 30323 catalog_manager.cc:4841] TS bc6e8671ca9b4737b7751a2db517b6bf (127.27.77.195:37977): tablet 7ef79abc61f7467db46f44bb6415738b (table test-table [id=32714b9aa4334c38a153f7a0769f7ef2]) successfully deleted
I20260806 00:53:50.819963 31077 raft_consensus.cc:729] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755b7d50fe404f5e99a93cda6285c0e5, State: Initialized, Role: FOLLOWER
I20260806 00:53:50.820092 31077 consensus_queue.cc:260] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.820345 31077 ts_tablet_manager.cc:1428] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:53:50.821931 31078 tablet_bootstrap.cc:492] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf: No bootstrap required, opened a new log
I20260806 00:53:50.822021 31078 ts_tablet_manager.cc:1397] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260806 00:53:50.822465 31078 raft_consensus.cc:348] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.822567 31078 raft_consensus.cc:374] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:50.822598 31078 raft_consensus.cc:729] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6e8671ca9b4737b7751a2db517b6bf, State: Initialized, Role: FOLLOWER
I20260806 00:53:50.822728 31078 consensus_queue.cc:260] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:50.822964 31078 ts_tablet_manager.cc:1428] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:53:50.823958 31144 raft_consensus.cc:2258] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:50.826375 31144 ts_tablet_manager.cc:1905] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:50.830572 31144 ts_tablet_manager.cc:1918] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:53:50.830694 31144 log.cc:1196] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/7ef79abc61f7467db46f44bb6415738b
I20260806 00:53:50.831035 31144 ts_tablet_manager.cc:1939] T 7ef79abc61f7467db46f44bb6415738b P bff4d7d0085e4c7f921fd3502c0374eb: Deleting consensus metadata
I20260806 00:53:50.833549 30329 catalog_manager.cc:4841] TS bff4d7d0085e4c7f921fd3502c0374eb (127.27.77.194:38375): tablet 7ef79abc61f7467db46f44bb6415738b (table test-table [id=32714b9aa4334c38a153f7a0769f7ef2]) successfully deleted
I20260806 00:53:51.017139 31087 raft_consensus.cc:482] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:53:51.017289 31087 raft_consensus.cc:504] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:51.018486 31087 leader_election.cc:288] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 755b7d50fe404f5e99a93cda6285c0e5 (127.27.77.193:44301), bc6e8671ca9b4737b7751a2db517b6bf (127.27.77.195:37977)
I20260806 00:53:51.022770 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:51.018913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:53133 (local address 127.27.77.195:37977)
0806 00:53:51.019064 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:51.019071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:51.019294 (+ 223us) server_negotiation.cc:407] Connection header received
0806 00:53:51.019397 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:51.019401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:51.019472 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:51.019578 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:51.019902 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.020514 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.021709 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.022159 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.022254 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:51.022300 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:51.022538 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 00:53:51.022632 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 00:53:51.022812 31151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:51.018775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:37977 (local address 127.27.77.194:53133)
0806 00:53:51.019196 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:51.019208 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:51.019309 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:51.019634 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:51.019637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:51.019660 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:51.019783 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:51.019793 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.020650 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.020654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:51.021573 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:51.021584 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.022298 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.022304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:51.022403 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:51.022429 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:53:51.022514 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 00:53:51.022605 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":3,"thread_start_us":77,"threads_started":1}
I20260806 00:53:51.022853 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:51.018914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:55501 (local address 127.27.77.193:44301)
0806 00:53:51.019062 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:51.019068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:51.019500 (+ 432us) server_negotiation.cc:407] Connection header received
0806 00:53:51.019659 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:51.019661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:51.019713 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:51.019805 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:53:51.020145 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.020707 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.021824 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.022279 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.022397 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:51.022453 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:51.022647 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 00:53:51.022737 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:53:51.022830 31153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:51.018775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:44301 (local address 127.27.77.194:55501)
0806 00:53:51.019382 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:51.019397 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:51.019546 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:51.019828 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:51.019830 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:51.019866 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:53:51.019961 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:51.019967 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.020873 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.020876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:51.021669 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:51.021682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.022407 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.022410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:51.022505 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:51.022531 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:53:51.022610 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 00:53:51.022672 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":50,"thread_start_us":79,"threads_started":1}
I20260806 00:53:51.023506 30575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e786dfd3aae04ffcb270222627ca67c6" candidate_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "755b7d50fe404f5e99a93cda6285c0e5" is_pre_election: true
I20260806 00:53:51.023615 30971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e786dfd3aae04ffcb270222627ca67c6" candidate_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" is_pre_election: true
I20260806 00:53:51.023702 30575 raft_consensus.cc:2454] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bff4d7d0085e4c7f921fd3502c0374eb in term 0.
I20260806 00:53:51.023759 30971 raft_consensus.cc:2454] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bff4d7d0085e4c7f921fd3502c0374eb in term 0.
I20260806 00:53:51.024097 30683 leader_election.cc:302] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755b7d50fe404f5e99a93cda6285c0e5, bff4d7d0085e4c7f921fd3502c0374eb; no voters:
I20260806 00:53:51.024405 31087 raft_consensus.cc:2781] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:53:51.024477 31087 raft_consensus.cc:482] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:53:51.024518 31087 raft_consensus.cc:3037] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:51.025816 31087 raft_consensus.cc:504] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:51.026181 31087 leader_election.cc:288] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [CANDIDATE]: Term 1 election: Requested vote from peers 755b7d50fe404f5e99a93cda6285c0e5 (127.27.77.193:44301), bc6e8671ca9b4737b7751a2db517b6bf (127.27.77.195:37977)
I20260806 00:53:51.026558 30575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e786dfd3aae04ffcb270222627ca67c6" candidate_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "755b7d50fe404f5e99a93cda6285c0e5"
I20260806 00:53:51.026698 30575 raft_consensus.cc:3037] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:51.026680 30971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e786dfd3aae04ffcb270222627ca67c6" candidate_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6e8671ca9b4737b7751a2db517b6bf"
I20260806 00:53:51.026832 30971 raft_consensus.cc:3037] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:51.027771 30575 raft_consensus.cc:2454] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff4d7d0085e4c7f921fd3502c0374eb in term 1.
I20260806 00:53:51.028019 30971 raft_consensus.cc:2454] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff4d7d0085e4c7f921fd3502c0374eb in term 1.
I20260806 00:53:51.028170 30683 leader_election.cc:302] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755b7d50fe404f5e99a93cda6285c0e5, bff4d7d0085e4c7f921fd3502c0374eb; no voters:
I20260806 00:53:51.028366 31087 raft_consensus.cc:2781] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:51.028586 31087 raft_consensus.cc:686] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [term 1 LEADER]: Becoming Leader. State: Replica: bff4d7d0085e4c7f921fd3502c0374eb, State: Running, Role: LEADER
I20260806 00:53:51.028859 31087 consensus_queue.cc:237] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:51.031340 30354 catalog_manager.cc:5495] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb reported cstate change: term changed from 0 to 1, leader changed from <none> to bff4d7d0085e4c7f921fd3502c0374eb (127.27.77.194). New cstate: current_term: 1 leader_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } health_report { overall_health: UNKNOWN } } }
I20260806 00:53:51.073184 31073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:51.069713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59886 (local address 127.27.77.194:38375)
0806 00:53:51.069834 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:51.069838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:51.069851 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:53:51.069993 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:51.069995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:51.070051 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:51.070174 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:51.070417 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.070908 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.071856 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.072087 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.072173 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:51.072382 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:51.072831 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:51.072921 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:51.072978 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 00:53:51.073070 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:53:51.084858 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:51.081249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:42205 (local address 127.27.77.193:44301)
0806 00:53:51.081413 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:51.081418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:51.081601 (+ 183us) server_negotiation.cc:407] Connection header received
0806 00:53:51.081771 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:51.081776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:51.081831 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:51.081911 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:51.082239 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.082747 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.083691 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.083912 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.084025 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:51.084399 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:51.084471 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:51.084552 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:51.084650 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 00:53:51.084701 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:53:51.084925 31159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:51.081105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:44301 (local address 127.27.77.194:42205)
0806 00:53:51.081494 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:51.081507 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:51.081658 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:51.081934 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:51.081937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:51.081945 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:53:51.082117 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:51.082124 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.082860 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.082865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:51.083507 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:51.083516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.084076 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.084081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:51.084183 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:51.084229 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:53:51.084558 (+ 329us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:53:51.084566 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 00:53:51.084630 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 00:53:51.084713 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":120,"threads_started":1}
I20260806 00:53:51.092307 30575 raft_consensus.cc:1261] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Refusing update from remote peer bff4d7d0085e4c7f921fd3502c0374eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:51.092377 30971 raft_consensus.cc:1261] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Refusing update from remote peer bff4d7d0085e4c7f921fd3502c0374eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:51.092940 31154 consensus_queue.cc:1035] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:51.093194 31087 consensus_queue.cc:1035] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:51.112576 31137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:51.108907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:38375 (local address 127.27.77.193:38373)
0806 00:53:51.109129 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:51.109147 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:51.109252 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:51.109556 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:51.109559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:51.109568 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:53:51.109691 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:51.109696 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.110399 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.110401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:51.111141 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:51.111152 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.111719 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.111723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:51.111824 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:51.111859 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:53:51.112223 (+ 364us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:53:51.112232 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 00:53:51.112319 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 00:53:51.112423 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260806 00:53:51.112567 31073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:51.108985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:38373 (local address 127.27.77.194:38375)
0806 00:53:51.109121 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:51.109126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:51.109195 (+ 69us) server_negotiation.cc:407] Connection header received
0806 00:53:51.109393 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:51.109396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:51.109450 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:51.109510 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:51.109768 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.110310 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.111265 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:51.111581 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:51.111712 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:51.112000 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:51.112123 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:51.112202 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:51.112348 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 00:53:51.112414 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 00:53:51.113456 31118 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 00:53:52.675799 30342 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:53:52.675985 30342 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:53:52.678718 30342 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2: Sending DeleteTablet for 3 replicas of tablet e786dfd3aae04ffcb270222627ca67c6
I20260806 00:53:52.679386 30750 tablet_service.cc:1514] Processing DeleteTablet for tablet e786dfd3aae04ffcb270222627ca67c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:52 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:59876
I20260806 00:53:52.679409 30549 tablet_service.cc:1514] Processing DeleteTablet for tablet e786dfd3aae04ffcb270222627ca67c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:52 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:50254
I20260806 00:53:52.679498 30945 tablet_service.cc:1514] Processing DeleteTablet for tablet e786dfd3aae04ffcb270222627ca67c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:52 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:37018
I20260806 00:53:52.679896 31198 tablet_replica.cc:330] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf: stopping tablet replica
I20260806 00:53:52.680043 31197 tablet_replica.cc:330] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5: stopping tablet replica
I20260806 00:53:52.680044 31198 raft_consensus.cc:2229] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:53:52.680181 31198 raft_consensus.cc:2258] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:52.680176 31197 raft_consensus.cc:2229] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:53:52.680289 31197 raft_consensus.cc:2258] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:52.680459 31196 tablet_replica.cc:330] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb: stopping tablet replica
I20260806 00:53:52.680598 31196 raft_consensus.cc:2229] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:53:52.680891 31198 ts_tablet_manager.cc:1905] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:52.680943 31196 raft_consensus.cc:2258] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:52.681334 31197 ts_tablet_manager.cc:1905] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:52.681561 31196 ts_tablet_manager.cc:1905] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:52.683974 31198 ts_tablet_manager.cc:1918] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 00:53:52.684095 31198 log.cc:1196] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/e786dfd3aae04ffcb270222627ca67c6
I20260806 00:53:52.684427 31198 ts_tablet_manager.cc:1939] T e786dfd3aae04ffcb270222627ca67c6 P bc6e8671ca9b4737b7751a2db517b6bf: Deleting consensus metadata
I20260806 00:53:52.684417 31196 ts_tablet_manager.cc:1918] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 00:53:52.684491 31196 log.cc:1196] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/e786dfd3aae04ffcb270222627ca67c6
I20260806 00:53:52.684785 31196 ts_tablet_manager.cc:1939] T e786dfd3aae04ffcb270222627ca67c6 P bff4d7d0085e4c7f921fd3502c0374eb: Deleting consensus metadata
I20260806 00:53:52.685120 31197 ts_tablet_manager.cc:1918] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 00:53:52.685259 31197 log.cc:1196] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/wal/wals/e786dfd3aae04ffcb270222627ca67c6
I20260806 00:53:52.685240 30323 catalog_manager.cc:4841] TS bc6e8671ca9b4737b7751a2db517b6bf (127.27.77.195:37977): tablet e786dfd3aae04ffcb270222627ca67c6 (table test-table [id=2bef6463fe9e48ad8cabc23830b8f5d3]) successfully deleted
I20260806 00:53:52.685492 30329 catalog_manager.cc:4841] TS bff4d7d0085e4c7f921fd3502c0374eb (127.27.77.194:38375): tablet e786dfd3aae04ffcb270222627ca67c6 (table test-table [id=2bef6463fe9e48ad8cabc23830b8f5d3]) successfully deleted
I20260806 00:53:52.685595 31197 ts_tablet_manager.cc:1939] T e786dfd3aae04ffcb270222627ca67c6 P 755b7d50fe404f5e99a93cda6285c0e5: Deleting consensus metadata
I20260806 00:53:52.686295 30329 catalog_manager.cc:4841] TS 755b7d50fe404f5e99a93cda6285c0e5 (127.27.77.193:44301): tablet e786dfd3aae04ffcb270222627ca67c6 (table test-table [id=2bef6463fe9e48ad8cabc23830b8f5d3]) successfully deleted
Aug 06 00:53:52 dist-test-slave-sb2z krb5kdc[30268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977632, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 00:53:52 dist-test-slave-sb2z krb5kdc[30268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977632, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:52.716615 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:52.702238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47384 (local address 127.27.77.254:43717)
0806 00:53:52.702538 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:52.702571 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:52.702652 (+ 81us) server_negotiation.cc:407] Connection header received
0806 00:53:52.702847 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:52.702849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:52.702897 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:52.702996 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:52.703823 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:52.704390 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:52.705293 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:52.705526 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:52.705647 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:52.712749 (+ 7102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:52.712753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:52.712755 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:52.712791 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:52.714488 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:52.715095 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:52.715099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:52.715100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:52.715193 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:52.715478 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:52.715481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:52.715482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:52.715817 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:52.715947 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:52.716302 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 00:53:52.716445 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":118,"threads_started":1}
I20260806 00:53:52.719779 30342 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:53:52.720228 30342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:53:52.725485 30549 tablet_service.cc:1467] Processing CreateTablet for tablet 0116e1e4de4445de95499a19854f45df (DEFAULT_TABLE table=test-table [id=bf14d1d31c614c0bb28151b5321e0656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:52.725512 30750 tablet_service.cc:1467] Processing CreateTablet for tablet 0116e1e4de4445de95499a19854f45df (DEFAULT_TABLE table=test-table [id=bf14d1d31c614c0bb28151b5321e0656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:52.725720 30945 tablet_service.cc:1467] Processing CreateTablet for tablet 0116e1e4de4445de95499a19854f45df (DEFAULT_TABLE table=test-table [id=bf14d1d31c614c0bb28151b5321e0656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:52.725864 30549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0116e1e4de4445de95499a19854f45df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:52.725871 30750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0116e1e4de4445de95499a19854f45df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:52.726013 30945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0116e1e4de4445de95499a19854f45df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:52.728639 31213 tablet_bootstrap.cc:492] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf: Bootstrap starting.
I20260806 00:53:52.728822 31212 tablet_bootstrap.cc:492] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb: Bootstrap starting.
I20260806 00:53:52.729794 31213 tablet_bootstrap.cc:654] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:52.729825 31212 tablet_bootstrap.cc:654] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:52.731172 31212 tablet_bootstrap.cc:492] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb: No bootstrap required, opened a new log
I20260806 00:53:52.731238 31213 tablet_bootstrap.cc:492] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf: No bootstrap required, opened a new log
I20260806 00:53:52.731268 31212 ts_tablet_manager.cc:1397] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:53:52.731338 31213 ts_tablet_manager.cc:1397] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:53:52.731904 31212 raft_consensus.cc:348] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.731884 31213 raft_consensus.cc:348] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.732023 31212 raft_consensus.cc:374] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:52.732036 31213 raft_consensus.cc:374] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:52.732061 31213 raft_consensus.cc:729] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6e8671ca9b4737b7751a2db517b6bf, State: Initialized, Role: FOLLOWER
I20260806 00:53:52.732055 31212 raft_consensus.cc:729] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff4d7d0085e4c7f921fd3502c0374eb, State: Initialized, Role: FOLLOWER
I20260806 00:53:52.732187 31212 consensus_queue.cc:260] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.732213 31213 consensus_queue.cc:260] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.732455 31212 ts_tablet_manager.cc:1428] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:53:52.732650 31213 ts_tablet_manager.cc:1428] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:53:52.733744 31211 tablet_bootstrap.cc:492] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5: Bootstrap starting.
I20260806 00:53:52.735114 31211 tablet_bootstrap.cc:654] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:52.737078 31211 tablet_bootstrap.cc:492] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5: No bootstrap required, opened a new log
I20260806 00:53:52.737185 31211 ts_tablet_manager.cc:1397] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 00:53:52.737537 31217 raft_consensus.cc:482] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:53:52.737648 31217 raft_consensus.cc:504] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.737658 31211 raft_consensus.cc:348] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.737749 31211 raft_consensus.cc:374] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:52.737771 31211 raft_consensus.cc:729] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755b7d50fe404f5e99a93cda6285c0e5, State: Initialized, Role: FOLLOWER
I20260806 00:53:52.737910 31211 consensus_queue.cc:260] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.738004 31217 leader_election.cc:288] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 755b7d50fe404f5e99a93cda6285c0e5 (127.27.77.193:44301), bff4d7d0085e4c7f921fd3502c0374eb (127.27.77.194:38375)
I20260806 00:53:52.738240 31211 ts_tablet_manager.cc:1428] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 00:53:52.738368 30575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0116e1e4de4445de95499a19854f45df" candidate_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "755b7d50fe404f5e99a93cda6285c0e5" is_pre_election: true
I20260806 00:53:52.738497 30575 raft_consensus.cc:2454] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc6e8671ca9b4737b7751a2db517b6bf in term 0.
I20260806 00:53:52.738466 30775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0116e1e4de4445de95499a19854f45df" candidate_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" is_pre_election: true
I20260806 00:53:52.738618 30775 raft_consensus.cc:2454] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc6e8671ca9b4737b7751a2db517b6bf in term 0.
I20260806 00:53:52.738862 30881 leader_election.cc:302] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755b7d50fe404f5e99a93cda6285c0e5, bc6e8671ca9b4737b7751a2db517b6bf; no voters:
I20260806 00:53:52.739071 31217 raft_consensus.cc:2781] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:53:52.739133 31217 raft_consensus.cc:482] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:53:52.739167 31217 raft_consensus.cc:3037] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:52.740461 31217 raft_consensus.cc:504] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.741101 30575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0116e1e4de4445de95499a19854f45df" candidate_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "755b7d50fe404f5e99a93cda6285c0e5"
I20260806 00:53:52.741225 30575 raft_consensus.cc:3037] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:52.741458 31217 leader_election.cc:288] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [CANDIDATE]: Term 1 election: Requested vote from peers 755b7d50fe404f5e99a93cda6285c0e5 (127.27.77.193:44301), bff4d7d0085e4c7f921fd3502c0374eb (127.27.77.194:38375)
I20260806 00:53:52.741840 30775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0116e1e4de4445de95499a19854f45df" candidate_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff4d7d0085e4c7f921fd3502c0374eb"
I20260806 00:53:52.741986 30775 raft_consensus.cc:3037] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:52.742363 30575 raft_consensus.cc:2454] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc6e8671ca9b4737b7751a2db517b6bf in term 1.
I20260806 00:53:52.742682 30881 leader_election.cc:302] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755b7d50fe404f5e99a93cda6285c0e5, bc6e8671ca9b4737b7751a2db517b6bf; no voters:
I20260806 00:53:52.742905 31217 raft_consensus.cc:2781] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:52.743223 31217 raft_consensus.cc:686] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [term 1 LEADER]: Becoming Leader. State: Replica: bc6e8671ca9b4737b7751a2db517b6bf, State: Running, Role: LEADER
I20260806 00:53:52.743496 30775 raft_consensus.cc:2454] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc6e8671ca9b4737b7751a2db517b6bf in term 1.
I20260806 00:53:52.743448 31217 consensus_queue.cc:237] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.745647 30342 catalog_manager.cc:5495] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf reported cstate change: term changed from 0 to 1, leader changed from <none> to bc6e8671ca9b4737b7751a2db517b6bf (127.27.77.195). New cstate: current_term: 1 leader_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } health_report { overall_health: HEALTHY } } }
I20260806 00:53:52.766770 31223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:52.763167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46170 (local address 127.27.77.195:37977)
0806 00:53:52.763495 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:52.763503 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:52.763516 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:53:52.763635 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:52.763638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:52.763682 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:52.763757 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:52.764143 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:52.764773 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:52.765683 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:52.765913 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:52.765999 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:52.766270 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:52.766348 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:52.766407 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:52.766498 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 00:53:52.766546 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":138,"threads_started":1}
I20260806 00:53:52.774302 30775 raft_consensus.cc:1261] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Refusing update from remote peer bc6e8671ca9b4737b7751a2db517b6bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:52.774278 30575 raft_consensus.cc:1261] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Refusing update from remote peer bc6e8671ca9b4737b7751a2db517b6bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:52.774742 31219 consensus_queue.cc:1035] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:52.774947 31217 consensus_queue.cc:1035] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:52.810674 30354 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:53:52.810851 30354 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:53:52.813300 30354 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2: Sending DeleteTablet for 3 replicas of tablet 0116e1e4de4445de95499a19854f45df
I20260806 00:53:52.814085 30945 tablet_service.cc:1514] Processing DeleteTablet for tablet 0116e1e4de4445de95499a19854f45df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:52 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:37018
I20260806 00:53:52.814165 30750 tablet_service.cc:1514] Processing DeleteTablet for tablet 0116e1e4de4445de95499a19854f45df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:52 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:59876
I20260806 00:53:52.814312 31198 tablet_replica.cc:330] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf: stopping tablet replica
I20260806 00:53:52.814411 31196 tablet_replica.cc:330] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb: stopping tablet replica
I20260806 00:53:52.814451 31198 raft_consensus.cc:2229] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:53:52.814558 31196 raft_consensus.cc:2229] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:53:52.814721 31196 raft_consensus.cc:2258] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:52.814713 31198 raft_consensus.cc:2258] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:52.814849 30354 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 00:53:52.815111 30549 tablet_service.cc:1514] Processing DeleteTablet for tablet 0116e1e4de4445de95499a19854f45df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:52 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:50254
I20260806 00:53:52.815325 31197 tablet_replica.cc:330] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5: stopping tablet replica
W20260806 00:53:52.815352 30354 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:53:52.815469 31197 raft_consensus.cc:2229] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:53:52.815658 31198 ts_tablet_manager.cc:1905] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:52.815652 31197 raft_consensus.cc:2258] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:52.816255 31196 ts_tablet_manager.cc:1905] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:52.818254 31198 ts_tablet_manager.cc:1918] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:53:52.818368 31198 log.cc:1196] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/0116e1e4de4445de95499a19854f45df
I20260806 00:53:52.818651 31198 ts_tablet_manager.cc:1939] T 0116e1e4de4445de95499a19854f45df P bc6e8671ca9b4737b7751a2db517b6bf: Deleting consensus metadata
I20260806 00:53:52.819489 31196 ts_tablet_manager.cc:1918] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:53:52.819605 31196 log.cc:1196] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/0116e1e4de4445de95499a19854f45df
I20260806 00:53:52.819834 30323 catalog_manager.cc:4841] TS bc6e8671ca9b4737b7751a2db517b6bf (127.27.77.195:37977): tablet 0116e1e4de4445de95499a19854f45df (table test-table [id=bf14d1d31c614c0bb28151b5321e0656]) successfully deleted
I20260806 00:53:52.819902 31196 ts_tablet_manager.cc:1939] T 0116e1e4de4445de95499a19854f45df P bff4d7d0085e4c7f921fd3502c0374eb: Deleting consensus metadata
I20260806 00:53:52.820796 30329 catalog_manager.cc:4841] TS bff4d7d0085e4c7f921fd3502c0374eb (127.27.77.194:38375): tablet 0116e1e4de4445de95499a19854f45df (table test-table [id=bf14d1d31c614c0bb28151b5321e0656]) successfully deleted
I20260806 00:53:52.820981 31197 ts_tablet_manager.cc:1905] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:52.821084 30549 tablet_service.cc:1467] Processing CreateTablet for tablet edcf2f98a52b42dba135d79551492ccc (DEFAULT_TABLE table=test-table [id=c573d08bcff3403a8fa5fddf24adf654]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:52.821358 30549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edcf2f98a52b42dba135d79551492ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:52.821484 30750 tablet_service.cc:1467] Processing CreateTablet for tablet edcf2f98a52b42dba135d79551492ccc (DEFAULT_TABLE table=test-table [id=c573d08bcff3403a8fa5fddf24adf654]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:52.821750 30750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edcf2f98a52b42dba135d79551492ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:52.821723 30945 tablet_service.cc:1467] Processing CreateTablet for tablet edcf2f98a52b42dba135d79551492ccc (DEFAULT_TABLE table=test-table [id=c573d08bcff3403a8fa5fddf24adf654]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:53:52.821995 30945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edcf2f98a52b42dba135d79551492ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:52.823624 31197 ts_tablet_manager.cc:1918] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:53:52.823701 31197 log.cc:1196] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-0/wal/wals/0116e1e4de4445de95499a19854f45df
I20260806 00:53:52.823900 31197 ts_tablet_manager.cc:1939] T 0116e1e4de4445de95499a19854f45df P 755b7d50fe404f5e99a93cda6285c0e5: Deleting consensus metadata
I20260806 00:53:52.824260 31212 tablet_bootstrap.cc:492] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb: Bootstrap starting.
I20260806 00:53:52.824627 31213 tablet_bootstrap.cc:492] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf: Bootstrap starting.
I20260806 00:53:52.825320 31212 tablet_bootstrap.cc:654] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:52.825623 31213 tablet_bootstrap.cc:654] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:52.826040 30329 catalog_manager.cc:4841] TS 755b7d50fe404f5e99a93cda6285c0e5 (127.27.77.193:44301): tablet 0116e1e4de4445de95499a19854f45df (table test-table [id=bf14d1d31c614c0bb28151b5321e0656]) successfully deleted
I20260806 00:53:52.826303 31211 tablet_bootstrap.cc:492] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5: Bootstrap starting.
I20260806 00:53:52.826632 31212 tablet_bootstrap.cc:492] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb: No bootstrap required, opened a new log
I20260806 00:53:52.826769 31212 ts_tablet_manager.cc:1397] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:53:52.827232 31213 tablet_bootstrap.cc:492] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf: No bootstrap required, opened a new log
I20260806 00:53:52.827236 31212 raft_consensus.cc:348] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.827310 31211 tablet_bootstrap.cc:654] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:52.827334 31212 raft_consensus.cc:374] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:52.827327 31213 ts_tablet_manager.cc:1397] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:53:52.827356 31212 raft_consensus.cc:729] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff4d7d0085e4c7f921fd3502c0374eb, State: Initialized, Role: FOLLOWER
I20260806 00:53:52.827507 31212 consensus_queue.cc:260] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.827826 31212 ts_tablet_manager.cc:1428] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 00:53:52.827805 31213 raft_consensus.cc:348] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.827916 31213 raft_consensus.cc:374] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:52.827955 31213 raft_consensus.cc:729] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6e8671ca9b4737b7751a2db517b6bf, State: Initialized, Role: FOLLOWER
I20260806 00:53:52.828125 31213 consensus_queue.cc:260] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.828594 31211 tablet_bootstrap.cc:492] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5: No bootstrap required, opened a new log
I20260806 00:53:52.828685 31211 ts_tablet_manager.cc:1397] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260806 00:53:52.829082 31213 ts_tablet_manager.cc:1428] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 00:53:52.829077 31211 raft_consensus.cc:348] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.829205 31211 raft_consensus.cc:374] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:52.829262 31211 raft_consensus.cc:729] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755b7d50fe404f5e99a93cda6285c0e5, State: Initialized, Role: FOLLOWER
I20260806 00:53:52.829394 31211 consensus_queue.cc:260] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.829726 31211 ts_tablet_manager.cc:1428] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:53:52.964110 31110 raft_consensus.cc:482] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:53:52.964265 31110 raft_consensus.cc:504] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.964763 31110 leader_election.cc:288] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bff4d7d0085e4c7f921fd3502c0374eb (127.27.77.194:38375), bc6e8671ca9b4737b7751a2db517b6bf (127.27.77.195:37977)
I20260806 00:53:52.965341 30775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edcf2f98a52b42dba135d79551492ccc" candidate_uuid: "755b7d50fe404f5e99a93cda6285c0e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" is_pre_election: true
I20260806 00:53:52.965341 30971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edcf2f98a52b42dba135d79551492ccc" candidate_uuid: "755b7d50fe404f5e99a93cda6285c0e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" is_pre_election: true
I20260806 00:53:52.965523 30775 raft_consensus.cc:2454] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 755b7d50fe404f5e99a93cda6285c0e5 in term 0.
I20260806 00:53:52.965533 30971 raft_consensus.cc:2454] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 755b7d50fe404f5e99a93cda6285c0e5 in term 0.
I20260806 00:53:52.965898 30489 leader_election.cc:302] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755b7d50fe404f5e99a93cda6285c0e5, bff4d7d0085e4c7f921fd3502c0374eb; no voters:
I20260806 00:53:52.966109 31110 raft_consensus.cc:2781] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:53:52.966174 31110 raft_consensus.cc:482] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:53:52.966213 31110 raft_consensus.cc:3037] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:52.967329 31110 raft_consensus.cc:504] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.967716 31110 leader_election.cc:288] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [CANDIDATE]: Term 1 election: Requested vote from peers bff4d7d0085e4c7f921fd3502c0374eb (127.27.77.194:38375), bc6e8671ca9b4737b7751a2db517b6bf (127.27.77.195:37977)
I20260806 00:53:52.968148 30775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edcf2f98a52b42dba135d79551492ccc" candidate_uuid: "755b7d50fe404f5e99a93cda6285c0e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff4d7d0085e4c7f921fd3502c0374eb"
I20260806 00:53:52.968178 30971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edcf2f98a52b42dba135d79551492ccc" candidate_uuid: "755b7d50fe404f5e99a93cda6285c0e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6e8671ca9b4737b7751a2db517b6bf"
I20260806 00:53:52.968293 30971 raft_consensus.cc:3037] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:52.968294 30775 raft_consensus.cc:3037] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:52.969498 30775 raft_consensus.cc:2454] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 755b7d50fe404f5e99a93cda6285c0e5 in term 1.
I20260806 00:53:52.969501 30971 raft_consensus.cc:2454] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 755b7d50fe404f5e99a93cda6285c0e5 in term 1.
I20260806 00:53:52.969831 30487 leader_election.cc:302] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755b7d50fe404f5e99a93cda6285c0e5, bc6e8671ca9b4737b7751a2db517b6bf; no voters:
I20260806 00:53:52.970049 31110 raft_consensus.cc:2781] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:52.970150 31110 raft_consensus.cc:686] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [term 1 LEADER]: Becoming Leader. State: Replica: 755b7d50fe404f5e99a93cda6285c0e5, State: Running, Role: LEADER
I20260806 00:53:52.970293 31110 consensus_queue.cc:237] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } }
I20260806 00:53:52.972127 30342 catalog_manager.cc:5495] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 755b7d50fe404f5e99a93cda6285c0e5 (127.27.77.193). New cstate: current_term: 1 leader_uuid: "755b7d50fe404f5e99a93cda6285c0e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "755b7d50fe404f5e99a93cda6285c0e5" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 } health_report { overall_health: UNKNOWN } } }
I20260806 00:53:52.992880 31234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:52.989535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47616 (local address 127.27.77.193:44301)
0806 00:53:52.989852 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:52.989861 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:52.989881 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:53:52.989925 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:52.989927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:52.989968 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:52.990056 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:52.990324 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:52.990869 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:52.991783 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:52.992025 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:52.992145 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:52.992398 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:52.992480 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:52.992562 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:52.992661 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 00:53:52.992717 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":127,"threads_started":1}
I20260806 00:53:53.002103 31234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:52.998763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:41397 (local address 127.27.77.193:44301)
0806 00:53:52.998891 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:52.998894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:52.998990 (+ 96us) server_negotiation.cc:407] Connection header received
0806 00:53:52.999137 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:52.999140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:52.999182 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:52.999251 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:52.999460 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:53.000144 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:53.000979 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:53.001270 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:53.001398 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:53.001660 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:53.001729 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:53.001797 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:53.001902 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 00:53:53.001975 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":54}
I20260806 00:53:53.002133 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:52.998640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:44301 (local address 127.27.77.193:41397)
0806 00:53:52.998905 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:52.998918 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:52.999002 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:52.999273 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:52.999276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:52.999282 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:53:52.999366 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:52.999370 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:53.000245 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:53.000248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:53.000841 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:53.000849 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:53.001410 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:53.001414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:53.001487 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:53.001515 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:53:53.001818 (+ 303us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:53:53.001823 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 00:53:53.001879 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 00:53:53.001969 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":1,"thread_start_us":116,"threads_started":1}
I20260806 00:53:53.008742 30971 raft_consensus.cc:1261] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Refusing update from remote peer 755b7d50fe404f5e99a93cda6285c0e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:53.008761 30775 raft_consensus.cc:1261] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Refusing update from remote peer 755b7d50fe404f5e99a93cda6285c0e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:53.009132 31172 consensus_queue.cc:1035] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff4d7d0085e4c7f921fd3502c0374eb" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 38375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:53.009306 31110 consensus_queue.cc:1035] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6e8671ca9b4737b7751a2db517b6bf" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 00:53:53.023581 31167 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 00:53:54.590512 30342 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:53:54.590668 30342 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:53:54.593289 30342 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9bc26f48adbd4a29b10ff375d41ebcf2: Sending DeleteTablet for 3 replicas of tablet edcf2f98a52b42dba135d79551492ccc
I20260806 00:53:54.593904 30549 tablet_service.cc:1514] Processing DeleteTablet for tablet edcf2f98a52b42dba135d79551492ccc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:54 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:50254
I20260806 00:53:54.594028 30750 tablet_service.cc:1514] Processing DeleteTablet for tablet edcf2f98a52b42dba135d79551492ccc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:54 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:59876
I20260806 00:53:54.594076 30945 tablet_service.cc:1514] Processing DeleteTablet for tablet edcf2f98a52b42dba135d79551492ccc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:53:54 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:37018
I20260806 00:53:54.594365 31274 tablet_replica.cc:330] T edcf2f98a52b42dba135d79551492ccc P 755b7d50fe404f5e99a93cda6285c0e5: stopping tablet replica
I20260806 00:53:54.594427 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 30462
I20260806 00:53:54.594515 31275 tablet_replica.cc:330] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb: stopping tablet replica
I20260806 00:53:54.605494 31276 tablet_replica.cc:330] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf: stopping tablet replica
I20260806 00:53:54.605499 31275 raft_consensus.cc:2229] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:53:54.605618 31276 raft_consensus.cc:2229] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:53:54.605660 31275 raft_consensus.cc:2258] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:54.605770 31276 raft_consensus.cc:2258] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:53:54.606961 31275 ts_tablet_manager.cc:1905] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:54.607259 31276 ts_tablet_manager.cc:1905] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:53:54.609339 31275 ts_tablet_manager.cc:1918] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 00:53:54.609452 31275 log.cc:1196] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/edcf2f98a52b42dba135d79551492ccc
I20260806 00:53:54.609743 31275 ts_tablet_manager.cc:1939] T edcf2f98a52b42dba135d79551492ccc P bff4d7d0085e4c7f921fd3502c0374eb: Deleting consensus metadata
I20260806 00:53:54.610525 30329 catalog_manager.cc:4841] TS bff4d7d0085e4c7f921fd3502c0374eb (127.27.77.194:38375): tablet edcf2f98a52b42dba135d79551492ccc (table test-table [id=c573d08bcff3403a8fa5fddf24adf654]) successfully deleted
I20260806 00:53:54.610886 31276 ts_tablet_manager.cc:1918] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 00:53:54.610985 31276 log.cc:1196] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/edcf2f98a52b42dba135d79551492ccc
I20260806 00:53:54.611282 31276 ts_tablet_manager.cc:1939] T edcf2f98a52b42dba135d79551492ccc P bc6e8671ca9b4737b7751a2db517b6bf: Deleting consensus metadata
I20260806 00:53:54.612116 30323 catalog_manager.cc:4841] TS bc6e8671ca9b4737b7751a2db517b6bf (127.27.77.195:37977): tablet edcf2f98a52b42dba135d79551492ccc (table test-table [id=c573d08bcff3403a8fa5fddf24adf654]) successfully deleted
W20260806 00:53:54.616446 30329 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:44301 (error 108)
I20260806 00:53:54.616670 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 30660
I20260806 00:53:54.623839 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:54.623180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:44301 (local address 127.0.0.1:47626)
0806 00:53:54.623496 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:54.623696 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:44301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":84,"threads_started":1}
W20260806 00:53:54.624023 30329 catalog_manager.cc:4584] TS 755b7d50fe404f5e99a93cda6285c0e5 (127.27.77.193:44301): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet edcf2f98a52b42dba135d79551492ccc: Network error: Client connection negotiation failed: client connection to 127.27.77.193:44301: connect: Connection refused (error 111)
I20260806 00:53:54.630520 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 30853
I20260806 00:53:54.643229 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 30288
2026-08-06T00:53:54Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6756 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:54 dist-test-slave-sb2z krb5kdc[31285](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:54 dist-test-slave-sb2z krb5kdc[31285](info): set up 2 sockets
Aug 06 00:53:54 dist-test-slave-sb2z krb5kdc[31285](info): commencing operation
krb5kdc: starting...
W20260806 00:53:55.880172 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:53:55 dist-test-slave-sb2z krb5kdc[31285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:53:56.015298 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:39951
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:42667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:39951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:56.131383 31300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:56.131776 31300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:56.131868 31300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:56.136096 31300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:56.136204 31300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:56.136241 31300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:56.136284 31300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:56.136325 31300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:56.142783 31300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:42667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:39951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:39951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:56.144109 31300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:56.145255 31300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:56.154876 31308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:56.154881 31310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:56.154947 31300 server_base.cc:1034] running on GCE node
W20260806 00:53:56.154876 31312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:56.155848 31300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:56.157018 31300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:56.158289 31300 hybrid_clock.cc:648] HybridClock initialized: now 1785977636158276 us; error 27 us; skew 500 ppm
Aug 06 00:53:56 dist-test-slave-sb2z krb5kdc[31285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977636, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:56.161536 31300 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:53:56.162588 31300 webserver.cc:460] Webserver started at http://127.27.77.254:38597/ using document root <none> and password file <none>
I20260806 00:53:56.163224 31300 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:56.163329 31300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:56.163638 31300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:56.165822 31300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "830b7ff1229144629f1fdc29d1b887e7"
format_stamp: "Formatted at 2026-08-06 00:53:56 on dist-test-slave-sb2z"
server_key: "9e0cbb4a1f548ce441c6e6a725d1629d"
server_key_iv: "9286992c9f2cd5e2e5ded87951ca3a13"
server_key_version: "encryptionkey@0"
I20260806 00:53:56.166419 31300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "830b7ff1229144629f1fdc29d1b887e7"
format_stamp: "Formatted at 2026-08-06 00:53:56 on dist-test-slave-sb2z"
server_key: "9e0cbb4a1f548ce441c6e6a725d1629d"
server_key_iv: "9286992c9f2cd5e2e5ded87951ca3a13"
server_key_version: "encryptionkey@0"
I20260806 00:53:56.170547 31300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 00:53:56.173390 31322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:56.174844 31300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:53:56.175014 31300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "830b7ff1229144629f1fdc29d1b887e7"
format_stamp: "Formatted at 2026-08-06 00:53:56 on dist-test-slave-sb2z"
server_key: "9e0cbb4a1f548ce441c6e6a725d1629d"
server_key_iv: "9286992c9f2cd5e2e5ded87951ca3a13"
server_key_version: "encryptionkey@0"
I20260806 00:53:56.175154 31300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:56.188226 31300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:56.191776 31300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:56.191998 31300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:56.199498 31300 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:39951
I20260806 00:53:56.199549 31395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:39951 every 8 connection(s)
I20260806 00:53:56.200558 31300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:56.203279 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 31300
I20260806 00:53:56.203375 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:53:56.203369 31397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:56.203706 27959 external_mini_cluster.cc:1428] Setting key b4269160357ea6ce6beccc8d0ffb48b7
I20260806 00:53:56.210393 31397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7: Bootstrap starting.
I20260806 00:53:56.213326 31397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7: Neither blocks nor log segments found. Creating new log.
Aug 06 00:53:56 dist-test-slave-sb2z krb5kdc[31285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:56.214147 31397 log.cc:824] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:56.216153 31397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7: No bootstrap required, opened a new log
I20260806 00:53:56.219004 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:56.205332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57636 (local address 127.27.77.254:39951)
0806 00:53:56.205765 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:56.205790 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:56.205825 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:53:56.206628 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:56.206649 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:56.207025 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:56.207309 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:56.208323 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:56.209362 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:56.210578 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:56.211059 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:56.211197 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:56.214203 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:56.214208 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:56.214247 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:56.214286 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:56.216175 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:56.216661 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:56.216667 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:56.216671 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:56.216783 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:56.217069 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:56.217073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:56.217075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:56.217477 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:56.217693 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:56.217988 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 00:53:56.218300 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":151,"threads_started":1}
I20260806 00:53:56.219878 31397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830b7ff1229144629f1fdc29d1b887e7" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39951 } }
I20260806 00:53:56.220139 31397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:56.220197 31397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 830b7ff1229144629f1fdc29d1b887e7, State: Initialized, Role: FOLLOWER
I20260806 00:53:56.220661 31397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830b7ff1229144629f1fdc29d1b887e7" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39951 } }
I20260806 00:53:56.220815 31397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:56.220876 31397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:56.220993 31397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:56.222018 31397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830b7ff1229144629f1fdc29d1b887e7" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39951 } }
I20260806 00:53:56.222308 31397 leader_election.cc:302] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 830b7ff1229144629f1fdc29d1b887e7; no voters:
I20260806 00:53:56.222568 31397 leader_election.cc:288] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:56.222704 31403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:56.222991 31403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [term 1 LEADER]: Becoming Leader. State: Replica: 830b7ff1229144629f1fdc29d1b887e7, State: Running, Role: LEADER
I20260806 00:53:56.223392 31403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830b7ff1229144629f1fdc29d1b887e7" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39951 } }
I20260806 00:53:56.223816 31397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:56.225178 31406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 830b7ff1229144629f1fdc29d1b887e7. Latest consensus state: current_term: 1 leader_uuid: "830b7ff1229144629f1fdc29d1b887e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830b7ff1229144629f1fdc29d1b887e7" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39951 } } }
I20260806 00:53:56.225170 31404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "830b7ff1229144629f1fdc29d1b887e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830b7ff1229144629f1fdc29d1b887e7" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39951 } } }
I20260806 00:53:56.225334 31404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:56.225334 31406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:56.225855 31416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:56.229164 31416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:56.234933 31416 catalog_manager.cc:1347] Generated new cluster ID: 32e22e62447b40d98b3428916804c2b5
I20260806 00:53:56.235013 31416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:56.255264 31416 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:56.256331 31416 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:56.264992 31416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 830b7ff1229144629f1fdc29d1b887e7: Generated new TSK 0
I20260806 00:53:56.265767 31416 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:53:56.303736 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39951
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:42667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:56.427371 31432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:56.427654 31432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:56.427697 31432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:56.431710 31432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:56.431766 31432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:56.431823 31432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:53:56.437033 31432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:42667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:56.438032 31432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:56.439034 31432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:56.449640 31442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:56.449605 31446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:56.449687 31432 server_base.cc:1034] running on GCE node
W20260806 00:53:56.449616 31443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:56.450397 31432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:56.451069 31432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:56.452299 31432 hybrid_clock.cc:648] HybridClock initialized: now 1785977636452282 us; error 28 us; skew 500 ppm
Aug 06 00:53:56 dist-test-slave-sb2z krb5kdc[31285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977636, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:56.455408 31432 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:53:56.456377 31432 webserver.cc:460] Webserver started at http://127.27.77.193:38181/ using document root <none> and password file <none>
I20260806 00:53:56.456974 31432 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:56.457083 31432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:56.457326 31432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:56.459578 31432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "e5818b3c22334565b0e9039fdf9a6980"
format_stamp: "Formatted at 2026-08-06 00:53:56 on dist-test-slave-sb2z"
server_key: "ed1adf3d5b670f5fa4097fb600086a5c"
server_key_iv: "58dc754556fd9031753b597cf3e9df17"
server_key_version: "encryptionkey@0"
I20260806 00:53:56.460194 31432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "e5818b3c22334565b0e9039fdf9a6980"
format_stamp: "Formatted at 2026-08-06 00:53:56 on dist-test-slave-sb2z"
server_key: "ed1adf3d5b670f5fa4097fb600086a5c"
server_key_iv: "58dc754556fd9031753b597cf3e9df17"
server_key_version: "encryptionkey@0"
I20260806 00:53:56.464242 31432 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:53:56.466837 31456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:56.468248 31432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:53:56.468398 31432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "e5818b3c22334565b0e9039fdf9a6980"
format_stamp: "Formatted at 2026-08-06 00:53:56 on dist-test-slave-sb2z"
server_key: "ed1adf3d5b670f5fa4097fb600086a5c"
server_key_iv: "58dc754556fd9031753b597cf3e9df17"
server_key_version: "encryptionkey@0"
I20260806 00:53:56.468523 31432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:56.499040 31432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:56.502753 31432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:56.502979 31432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:56.503701 31432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:53:56.504630 31432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:53:56.504707 31432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:56.504822 31432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:53:56.504868 31432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:56.514608 31432 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:35801
I20260806 00:53:56.514631 31626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:35801 every 8 connection(s)
I20260806 00:53:56.515601 31432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:53:56.522596 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 31432
I20260806 00:53:56.522722 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:53:56.523041 27959 external_mini_cluster.cc:1428] Setting key c730f517714d25758e23559c2a224076
Aug 06 00:53:56 dist-test-slave-sb2z krb5kdc[31285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977636, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:56.532389 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:56.517666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:35893 (local address 127.27.77.254:39951)
0806 00:53:56.517825 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:56.517830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:56.518846 (+ 1016us) server_negotiation.cc:407] Connection header received
0806 00:53:56.519905 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:56.519908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:56.519973 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:56.520069 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:56.521371 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:56.521985 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:56.523928 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:56.524325 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:56.524465 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:56.527646 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:56.527654 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:56.527659 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:56.527699 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:56.529337 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:56.530163 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:56.530168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:56.530172 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:56.530271 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:56.530739 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:56.530742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:56.530743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:56.531185 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:56.531286 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:56.532137 (+ 851us) server_negotiation.cc:299] Negotiation successful
0806 00:53:56.532257 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":59}
I20260806 00:53:56.533806 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:56.517956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:39951 (local address 127.27.77.193:35893)
0806 00:53:56.518677 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:56.518721 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:56.519683 (+ 962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:56.520259 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:56.520266 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:56.520660 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:56.521198 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:56.521214 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:56.522143 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:56.522152 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:56.523766 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:56.523778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:56.524542 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:56.524547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:56.524808 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:56.525564 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:56.525583 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:56.527414 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:56.529474 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:56.529485 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:56.529515 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:56.529995 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:56.530381 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:56.530387 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:56.530392 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:56.530624 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:56.531333 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:56.531344 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:56.531808 (+ 464us) client_negotiation.cc:769] Sending connection context
0806 00:53:56.532091 (+ 283us) client_negotiation.cc:240] Negotiation successful
0806 00:53:56.532555 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":128,"threads_started":1}
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
I20260806 00:53:56.535224 31627 heartbeater.cc:347] Connected to a master server at 127.27.77.254:39951
I20260806 00:53:56.535471 31627 heartbeater.cc:464] Registering TS with master...
I20260806 00:53:56.536042 31627 heartbeater.cc:511] Master 127.27.77.254:39951 requested a full tablet report, sending...
I20260806 00:53:56.537680 31348 ts_manager.cc:194] Registered new tserver with Master: e5818b3c22334565b0e9039fdf9a6980 (127.27.77.193:35801)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
I20260806 00:53:56.539001 31348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:35893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:53:56.558682 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39951
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:42667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:56.679994 31635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:56.680305 31635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:56.680399 31635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:56.684515 31635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:56.684598 31635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:56.684684 31635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:53:56.690141 31635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:42667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:56.691279 31635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:56.692377 31635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:53:56.702319 31635 server_base.cc:1034] running on GCE node
W20260806 00:53:56.702399 31640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:56.702351 31643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:56.702364 31641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:56.703051 31635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:56.703722 31635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:56.704905 31635 hybrid_clock.cc:648] HybridClock initialized: now 1785977636704871 us; error 44 us; skew 500 ppm
Aug 06 00:53:56 dist-test-slave-sb2z krb5kdc[31285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977636, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:56.708181 31635 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:53:56.709072 31635 webserver.cc:460] Webserver started at http://127.27.77.194:42425/ using document root <none> and password file <none>
I20260806 00:53:56.709684 31635 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:56.709786 31635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:56.710028 31635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:56.712179 31635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "d0b30d72056440bfb6d5038ab396f2e0"
format_stamp: "Formatted at 2026-08-06 00:53:56 on dist-test-slave-sb2z"
server_key: "eb99282706afb505d7ba0e7666629427"
server_key_iv: "d46f44463359fc7b727ede859fa7474f"
server_key_version: "encryptionkey@0"
I20260806 00:53:56.712751 31635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "d0b30d72056440bfb6d5038ab396f2e0"
format_stamp: "Formatted at 2026-08-06 00:53:56 on dist-test-slave-sb2z"
server_key: "eb99282706afb505d7ba0e7666629427"
server_key_iv: "d46f44463359fc7b727ede859fa7474f"
server_key_version: "encryptionkey@0"
I20260806 00:53:56.716535 31635 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:53:56.719130 31654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:56.720762 31635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:53:56.720903 31635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "d0b30d72056440bfb6d5038ab396f2e0"
format_stamp: "Formatted at 2026-08-06 00:53:56 on dist-test-slave-sb2z"
server_key: "eb99282706afb505d7ba0e7666629427"
server_key_iv: "d46f44463359fc7b727ede859fa7474f"
server_key_version: "encryptionkey@0"
I20260806 00:53:56.721062 31635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:56.742563 31635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:56.746277 31635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:56.746526 31635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:56.747213 31635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:53:56.748190 31635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:53:56.748266 31635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:56.748344 31635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:53:56.748389 31635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:56.758613 31635 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:45815
I20260806 00:53:56.758642 31811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:45815 every 8 connection(s)
I20260806 00:53:56.759716 31635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:53:56.767158 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 31635
I20260806 00:53:56.767309 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:53:56.767719 27959 external_mini_cluster.cc:1428] Setting key c1b3020d2c859f2ffd90245c4c48be0d
Aug 06 00:53:56 dist-test-slave-sb2z krb5kdc[31285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977636, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:56.777428 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:56.761884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:34259 (local address 127.27.77.254:39951)
0806 00:53:56.762074 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:56.762080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:56.763363 (+ 1283us) server_negotiation.cc:407] Connection header received
0806 00:53:56.764372 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:56.764375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:56.764440 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:56.764565 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:56.766044 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:56.766673 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:56.768542 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:56.768814 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:56.768946 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:56.772269 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:56.772276 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:56.772281 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:56.772323 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:56.774392 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:56.775013 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:56.775021 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:56.775026 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:56.775155 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:56.775664 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:56.775672 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:56.775676 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:56.776001 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:56.776131 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:56.776946 (+ 815us) server_negotiation.cc:299] Negotiation successful
0806 00:53:56.777141 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 00:53:56.778439 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:56.762296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:39951 (local address 127.27.77.194:34259)
0806 00:53:56.763127 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:56.763186 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:56.764082 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:56.764787 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:56.764800 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:56.765308 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:56.765873 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:56.765890 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:56.766835 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:56.766857 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:56.768383 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:56.768396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:56.768997 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:56.769001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:56.769202 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:56.769990 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:56.770021 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:56.772053 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:56.774533 (+ 2480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:56.774538 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:56.774553 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:56.774868 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:56.775267 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:56.775270 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:56.775271 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:56.775466 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:56.776309 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:56.776315 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:56.776617 (+ 302us) client_negotiation.cc:769] Sending connection context
0806 00:53:56.776805 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 00:53:56.777201 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":117,"threads_started":1}
I20260806 00:53:56.779846 31812 heartbeater.cc:347] Connected to a master server at 127.27.77.254:39951
I20260806 00:53:56.780130 31812 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
I20260806 00:53:56.780720 31812 heartbeater.cc:511] Master 127.27.77.254:39951 requested a full tablet report, sending...
I20260806 00:53:56.781886 31348 ts_manager.cc:194] Registered new tserver with Master: d0b30d72056440bfb6d5038ab396f2e0 (127.27.77.194:45815)
I20260806 00:53:56.782459 31348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:34259
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:53:56.803041 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39951
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:42667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:56.921473 31819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:56.921744 31819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:56.921789 31819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:56.926277 31819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:56.926347 31819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:56.926429 31819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:53:56.931816 31819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:42667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:56.932925 31819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:56.933971 31819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:56.943897 31833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:56.943893 31827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:56.943974 31819 server_base.cc:1034] running on GCE node
W20260806 00:53:56.943893 31828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:56.944651 31819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:56.945297 31819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:56.946496 31819 hybrid_clock.cc:648] HybridClock initialized: now 1785977636946458 us; error 48 us; skew 500 ppm
Aug 06 00:53:56 dist-test-slave-sb2z krb5kdc[31285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977636, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:56.949453 31819 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:53:56.950387 31819 webserver.cc:460] Webserver started at http://127.27.77.195:33529/ using document root <none> and password file <none>
I20260806 00:53:56.950991 31819 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:56.951090 31819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:56.951332 31819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:56.953521 31819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "a3b9612721e542c0ac87028a6b499e35"
format_stamp: "Formatted at 2026-08-06 00:53:56 on dist-test-slave-sb2z"
server_key: "4e11174a3fb5e57ddce1ba49876cf7db"
server_key_iv: "2f88d23feaebed830bbdef26d32f1dd1"
server_key_version: "encryptionkey@0"
I20260806 00:53:56.954123 31819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "a3b9612721e542c0ac87028a6b499e35"
format_stamp: "Formatted at 2026-08-06 00:53:56 on dist-test-slave-sb2z"
server_key: "4e11174a3fb5e57ddce1ba49876cf7db"
server_key_iv: "2f88d23feaebed830bbdef26d32f1dd1"
server_key_version: "encryptionkey@0"
I20260806 00:53:56.958041 31819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:53:56.960623 31848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:56.961869 31819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:53:56.962008 31819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "a3b9612721e542c0ac87028a6b499e35"
format_stamp: "Formatted at 2026-08-06 00:53:56 on dist-test-slave-sb2z"
server_key: "4e11174a3fb5e57ddce1ba49876cf7db"
server_key_iv: "2f88d23feaebed830bbdef26d32f1dd1"
server_key_version: "encryptionkey@0"
I20260806 00:53:56.962123 31819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:56.994185 31819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:56.997504 31819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:56.997741 31819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:56.998445 31819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:53:56.999351 31819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:53:56.999429 31819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:56.999501 31819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:53:56.999579 31819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 00:53:57.009291 31819 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:37759
I20260806 00:53:57.009315 32013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:37759 every 8 connection(s)
I20260806 00:53:57.010257 31819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:53:57.011422 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 31819
I20260806 00:53:57.011601 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:53:57.011973 27959 external_mini_cluster.cc:1428] Setting key 643b3d60159fcf57f6cb9063ad46ddf1
Aug 06 00:53:57 dist-test-slave-sb2z krb5kdc[31285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977636, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:57.025646 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:57.012601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:34829 (local address 127.27.77.254:39951)
0806 00:53:57.012763 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:57.012770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:57.014106 (+ 1336us) server_negotiation.cc:407] Connection header received
0806 00:53:57.015081 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:57.015084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:57.015133 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:57.015212 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:57.016485 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:57.017170 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:57.018327 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:57.018558 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:57.018647 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:57.021387 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:57.021390 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:57.021392 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:57.021419 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:57.023139 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:57.023768 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:57.023773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:57.023777 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:57.023869 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:57.024256 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:57.024259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:57.024260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:57.024510 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:57.024626 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:57.025320 (+ 694us) server_negotiation.cc:299] Negotiation successful
0806 00:53:57.025454 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:53:57.026631 32017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:57.013142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:39951 (local address 127.27.77.195:34829)
0806 00:53:57.013939 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:57.013996 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:57.014798 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:57.015386 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:57.015393 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:57.015844 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:57.016336 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:57.016350 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:57.017307 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:57.017310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:57.018203 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:57.018213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:57.018708 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:57.018711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:57.018864 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:57.019454 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:57.019499 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:57.021215 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:57.023269 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:57.023274 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:57.023288 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:57.023640 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:57.023954 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:57.023956 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:57.023957 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:57.024129 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:57.024651 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:57.024658 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:57.025044 (+ 386us) client_negotiation.cc:769] Sending connection context
0806 00:53:57.025253 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 00:53:57.025606 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":150,"threads_started":1}
I20260806 00:53:57.027829 32014 heartbeater.cc:347] Connected to a master server at 127.27.77.254:39951
I20260806 00:53:57.028060 32014 heartbeater.cc:464] Registering TS with master...
I20260806 00:53:57.028537 32014 heartbeater.cc:511] Master 127.27.77.254:39951 requested a full tablet report, sending...
I20260806 00:53:57.029572 31348 ts_manager.cc:194] Registered new tserver with Master: a3b9612721e542c0ac87028a6b499e35 (127.27.77.195:37759)
I20260806 00:53:57.030205 31348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:34829
I20260806 00:53:57.038614 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 00:53:57.054557 31401 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:57.053092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57638 (local address 127.27.77.254:39951)
0806 00:53:57.053254 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:57.053258 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:57.053398 (+ 140us) server_negotiation.cc:407] Connection header received
0806 00:53:57.053459 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:57.053461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:57.053511 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:57.053601 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:57.054429 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57638: BlockingRecv error: recv got EOF from 127.0.0.1:57638 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:53:57.055655 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 31432
I20260806 00:53:57.067919 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 31635
I20260806 00:53:57.076232 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 31819
I20260806 00:53:57.084173 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 31300
2026-08-06T00:53:57Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2434 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:57 dist-test-slave-sb2z krb5kdc[32031](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:57 dist-test-slave-sb2z krb5kdc[32031](info): set up 2 sockets
Aug 06 00:53:57 dist-test-slave-sb2z krb5kdc[32031](info): commencing operation
krb5kdc: starting...
W20260806 00:53:58.331215 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:53:58 dist-test-slave-sb2z krb5kdc[32031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:53:58.467515 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:34463
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:34463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:58.584563 32046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:58.584895 32046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:58.584973 32046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:58.589010 32046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:58.589108 32046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:58.589149 32046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:58.589211 32046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:58.589251 32046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:58.594594 32046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:34463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:34463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:58.595798 32046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:58.596928 32046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:58.606123 32053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:58.606120 32052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:58.606114 32055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:58.606395 32046 server_base.cc:1034] running on GCE node
I20260806 00:53:58.607103 32046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:58.608366 32046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:58.609598 32046 hybrid_clock.cc:648] HybridClock initialized: now 1785977638609577 us; error 35 us; skew 500 ppm
Aug 06 00:53:58 dist-test-slave-sb2z krb5kdc[32031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977638, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:58.615906 32046 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:53:58.616936 32046 webserver.cc:460] Webserver started at http://127.27.77.254:42907/ using document root <none> and password file <none>
I20260806 00:53:58.617625 32046 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:58.617777 32046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:58.618099 32046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:58.620675 32046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "5ee0ab6a1ac04543b1e9ed84ddeb3b59"
format_stamp: "Formatted at 2026-08-06 00:53:58 on dist-test-slave-sb2z"
server_key: "d6633928609dbeec96f5a71bf7034e97"
server_key_iv: "ba46fc9f00702fa7c94520468b35429b"
server_key_version: "encryptionkey@0"
I20260806 00:53:58.621265 32046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "5ee0ab6a1ac04543b1e9ed84ddeb3b59"
format_stamp: "Formatted at 2026-08-06 00:53:58 on dist-test-slave-sb2z"
server_key: "d6633928609dbeec96f5a71bf7034e97"
server_key_iv: "ba46fc9f00702fa7c94520468b35429b"
server_key_version: "encryptionkey@0"
I20260806 00:53:58.625435 32046 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:53:58.628091 32064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:58.629506 32046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:53:58.629699 32046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "5ee0ab6a1ac04543b1e9ed84ddeb3b59"
format_stamp: "Formatted at 2026-08-06 00:53:58 on dist-test-slave-sb2z"
server_key: "d6633928609dbeec96f5a71bf7034e97"
server_key_iv: "ba46fc9f00702fa7c94520468b35429b"
server_key_version: "encryptionkey@0"
I20260806 00:53:58.629808 32046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:58.656116 32046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:58.659734 32046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:58.659976 32046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:58.668134 32046 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:34463
I20260806 00:53:58.668438 32140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:34463 every 8 connection(s)
I20260806 00:53:58.669265 32046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:58.672263 32141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:58.675859 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 32046
I20260806 00:53:58.675997 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:53:58.676323 27959 external_mini_cluster.cc:1428] Setting key fc4913024ab794c6bcdf8d31dd2964bd
I20260806 00:53:58.679275 32141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59: Bootstrap starting.
I20260806 00:53:58.681658 32141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:58.682459 32141 log.cc:824] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:58.684504 32141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59: No bootstrap required, opened a new log
Aug 06 00:53:58 dist-test-slave-sb2z krb5kdc[32031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:53:58.688266 32141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee0ab6a1ac04543b1e9ed84ddeb3b59" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 34463 } }
I20260806 00:53:58.688587 32141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:58.688678 32141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ee0ab6a1ac04543b1e9ed84ddeb3b59, State: Initialized, Role: FOLLOWER
I20260806 00:53:58.689198 32141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee0ab6a1ac04543b1e9ed84ddeb3b59" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 34463 } }
I20260806 00:53:58.689381 32141 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:58.689487 32141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:58.689600 32141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:58.690511 32141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee0ab6a1ac04543b1e9ed84ddeb3b59" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 34463 } }
I20260806 00:53:58.690874 32141 leader_election.cc:302] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ee0ab6a1ac04543b1e9ed84ddeb3b59; no voters:
I20260806 00:53:58.691238 32141 leader_election.cc:288] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:58.691323 32149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:58.691578 32149 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [term 1 LEADER]: Becoming Leader. State: Replica: 5ee0ab6a1ac04543b1e9ed84ddeb3b59, State: Running, Role: LEADER
I20260806 00:53:58.691915 32149 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee0ab6a1ac04543b1e9ed84ddeb3b59" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 34463 } }
I20260806 00:53:58.692627 32141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:58.693192 32149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ee0ab6a1ac04543b1e9ed84ddeb3b59. Latest consensus state: current_term: 1 leader_uuid: "5ee0ab6a1ac04543b1e9ed84ddeb3b59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee0ab6a1ac04543b1e9ed84ddeb3b59" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 34463 } } }
I20260806 00:53:58.693397 32149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:58.694305 32146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:58.678336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54600 (local address 127.27.77.254:34463)
0806 00:53:58.678839 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:58.678860 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:58.678899 (+ 39us) server_negotiation.cc:407] Connection header received
0806 00:53:58.679755 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:58.679776 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:58.680146 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:58.680512 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:58.681509 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:58.682758 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:58.683796 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:58.684539 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:58.684748 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:58.687831 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:58.687846 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:58.687862 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:58.687892 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:58.690048 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:58.690544 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:58.690550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:58.690557 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:58.690731 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:58.691093 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:58.691097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:58.691099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:58.691721 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:58.692586 (+ 865us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:58.692787 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 00:53:58.693120 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":156,"threads_started":1}
I20260806 00:53:58.695802 32150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ee0ab6a1ac04543b1e9ed84ddeb3b59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee0ab6a1ac04543b1e9ed84ddeb3b59" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 34463 } } }
I20260806 00:53:58.695906 32150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:58.697868 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 00:53:58.698805 32170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:53:58.698895 32170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:53:58.698983 32171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:58.699980 32171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:58.706178 32171 catalog_manager.cc:1347] Generated new cluster ID: 686f376b638d4de28e10146ec90b9ed7
I20260806 00:53:58.706243 32171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:58.722910 32171 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:58.723194 32146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:58.716336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54614 (local address 127.27.77.254:34463)
0806 00:53:58.716450 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:58.716454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:58.716750 (+ 296us) server_negotiation.cc:407] Connection header received
0806 00:53:58.716885 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:58.716889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:58.716949 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:58.717052 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:58.717636 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:58.718599 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:58.719793 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:58.720206 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:58.720328 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:58.721373 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:58.721378 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:58.721381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:58.722015 (+ 634us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:58.722052 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:58.722076 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:58.722175 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:58.722311 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 00:53:58.722383 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":42}
I20260806 00:53:58.724243 32171 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:58.728744 32171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ee0ab6a1ac04543b1e9ed84ddeb3b59: Generated new TSK 0
I20260806 00:53:58.729595 32171 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:58.737241 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 32046
2026-08-06T00:53:58Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1656 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:58 dist-test-slave-sb2z krb5kdc[32188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:58 dist-test-slave-sb2z krb5kdc[32188](info): set up 2 sockets
Aug 06 00:53:58 dist-test-slave-sb2z krb5kdc[32188](info): commencing operation
krb5kdc: starting...
W20260806 00:53:59.946540 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:54:00 dist-test-slave-sb2z krb5kdc[32188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:54:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:54:00.082412 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:42729
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:42729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:00.197544 32208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:00.197871 32208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:00.197954 32208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:00.202071 32208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:00.202170 32208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:00.202205 32208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:00.202245 32208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:00.202278 32208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:54:00.207744 32208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:42729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:42729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:00.208873 32208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:00.209954 32208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:00.218796 32215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:00.218813 32219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:00.218791 32217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:00.219076 32208 server_base.cc:1034] running on GCE node
I20260806 00:54:00.219769 32208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:00.221177 32208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:00.222375 32208 hybrid_clock.cc:648] HybridClock initialized: now 1785977640222355 us; error 32 us; skew 500 ppm
Aug 06 00:54:00 dist-test-slave-sb2z krb5kdc[32188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977640, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:00.225366 32208 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:54:00.226258 32208 webserver.cc:460] Webserver started at http://127.27.77.254:33375/ using document root <none> and password file <none>
I20260806 00:54:00.226801 32208 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:00.226871 32208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:00.227056 32208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:00.229211 32208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "a21c09f97c154e35b653dc407cf5ea99"
format_stamp: "Formatted at 2026-08-06 00:54:00 on dist-test-slave-sb2z"
server_key: "ba522f983ed53f864c89ee0425d4e376"
server_key_iv: "9d949d44d249f280bc29034af086c629"
server_key_version: "encryptionkey@0"
I20260806 00:54:00.229746 32208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "a21c09f97c154e35b653dc407cf5ea99"
format_stamp: "Formatted at 2026-08-06 00:54:00 on dist-test-slave-sb2z"
server_key: "ba522f983ed53f864c89ee0425d4e376"
server_key_iv: "9d949d44d249f280bc29034af086c629"
server_key_version: "encryptionkey@0"
I20260806 00:54:00.233530 32208 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 00:54:00.235958 32232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:00.237237 32208 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:54:00.237370 32208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "a21c09f97c154e35b653dc407cf5ea99"
format_stamp: "Formatted at 2026-08-06 00:54:00 on dist-test-slave-sb2z"
server_key: "ba522f983ed53f864c89ee0425d4e376"
server_key_iv: "9d949d44d249f280bc29034af086c629"
server_key_version: "encryptionkey@0"
I20260806 00:54:00.237481 32208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:00.264590 32208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:00.267614 32208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:00.267833 32208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:00.275311 32208 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:42729
I20260806 00:54:00.275676 32297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:42729 every 8 connection(s)
I20260806 00:54:00.276490 32208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:00.279269 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:00.280092 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 32208
I20260806 00:54:00.280234 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:54:00.280555 27959 external_mini_cluster.cc:1428] Setting key 907805b214ff15ac66a3c42e0ffec95c
I20260806 00:54:00.285923 32298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99: Bootstrap starting.
I20260806 00:54:00.288700 32298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:00.289604 32298 log.cc:824] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:54:00 dist-test-slave-sb2z krb5kdc[32188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:00.292130 32298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99: No bootstrap required, opened a new log
I20260806 00:54:00.295893 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:00.282014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38992 (local address 127.27.77.254:42729)
0806 00:54:00.282436 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:00.282459 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:00.282495 (+ 36us) server_negotiation.cc:407] Connection header received
0806 00:54:00.283137 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:00.283153 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:00.283466 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:00.283815 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:00.284753 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:00.285772 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:00.286919 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:00.287390 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:00.287508 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:00.290565 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:00.290574 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:00.290594 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:00.290626 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:00.292660 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:00.293096 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:00.293102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:00.293108 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:00.293250 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:00.293507 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:00.293511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:00.293513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:00.294040 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:00.294261 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:00.294578 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 00:54:00.294906 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":136,"threads_started":1}
I20260806 00:54:00.297104 32298 raft_consensus.cc:348] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a21c09f97c154e35b653dc407cf5ea99" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42729 } }
I20260806 00:54:00.297418 32298 raft_consensus.cc:374] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:00.297484 32298 raft_consensus.cc:729] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a21c09f97c154e35b653dc407cf5ea99, State: Initialized, Role: FOLLOWER
I20260806 00:54:00.297996 32298 consensus_queue.cc:260] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a21c09f97c154e35b653dc407cf5ea99" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42729 } }
I20260806 00:54:00.298234 32298 raft_consensus.cc:388] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:00.298326 32298 raft_consensus.cc:482] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:00.298449 32298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:00.299880 32298 raft_consensus.cc:504] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a21c09f97c154e35b653dc407cf5ea99" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42729 } }
I20260806 00:54:00.300300 32298 leader_election.cc:302] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a21c09f97c154e35b653dc407cf5ea99; no voters:
I20260806 00:54:00.300695 32298 leader_election.cc:288] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:54:00.300848 32304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:00.301077 32304 raft_consensus.cc:686] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [term 1 LEADER]: Becoming Leader. State: Replica: a21c09f97c154e35b653dc407cf5ea99, State: Running, Role: LEADER
I20260806 00:54:00.301375 32304 consensus_queue.cc:237] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a21c09f97c154e35b653dc407cf5ea99" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42729 } }
I20260806 00:54:00.302155 32298 sys_catalog.cc:564] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:00.302654 32304 sys_catalog.cc:455] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a21c09f97c154e35b653dc407cf5ea99. Latest consensus state: current_term: 1 leader_uuid: "a21c09f97c154e35b653dc407cf5ea99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a21c09f97c154e35b653dc407cf5ea99" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42729 } } }
I20260806 00:54:00.302768 32304 sys_catalog.cc:458] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:00.303505 32309 sys_catalog.cc:455] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a21c09f97c154e35b653dc407cf5ea99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a21c09f97c154e35b653dc407cf5ea99" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42729 } } }
I20260806 00:54:00.303642 32309 sys_catalog.cc:458] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:00.305434 32311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:00.308736 32311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:00.314574 32311 catalog_manager.cc:1347] Generated new cluster ID: e792b8598c52467da4728f6ac3a93321
I20260806 00:54:00.314662 32311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:00.329082 32311 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:54:00.330287 32311 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:00.337450 32311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a21c09f97c154e35b653dc407cf5ea99: Generated new TSK 0
I20260806 00:54:00.338351 32311 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:54:00.374818 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:42729
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:00.494081 32334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:00.494351 32334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:00.494396 32334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:00.498512 32334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:00.498585 32334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:00.498646 32334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:54:00.504388 32334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:42729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:00.505560 32334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:00.506614 32334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:00.516395 32344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:00.516400 32347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:00.516426 32343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:00.516676 32334 server_base.cc:1034] running on GCE node
I20260806 00:54:00.517112 32334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:00.517807 32334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:00.518978 32334 hybrid_clock.cc:648] HybridClock initialized: now 1785977640518965 us; error 26 us; skew 500 ppm
Aug 06 00:54:00 dist-test-slave-sb2z krb5kdc[32188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977640, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:00.521914 32334 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:54:00.522814 32334 webserver.cc:460] Webserver started at http://127.27.77.193:41227/ using document root <none> and password file <none>
I20260806 00:54:00.523329 32334 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:00.523388 32334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:00.523641 32334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:00.525624 32334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "a010cd2ef63f4a57b0dea6598af2b8fb"
format_stamp: "Formatted at 2026-08-06 00:54:00 on dist-test-slave-sb2z"
server_key: "fdd3df2bbbe193992395d7b186621589"
server_key_iv: "853304f97783ff63b935068f9ddb0b97"
server_key_version: "encryptionkey@0"
I20260806 00:54:00.526140 32334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "a010cd2ef63f4a57b0dea6598af2b8fb"
format_stamp: "Formatted at 2026-08-06 00:54:00 on dist-test-slave-sb2z"
server_key: "fdd3df2bbbe193992395d7b186621589"
server_key_iv: "853304f97783ff63b935068f9ddb0b97"
server_key_version: "encryptionkey@0"
I20260806 00:54:00.529878 32334 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 00:54:00.532265 32354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:00.533521 32334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:54:00.533623 32334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "a010cd2ef63f4a57b0dea6598af2b8fb"
format_stamp: "Formatted at 2026-08-06 00:54:00 on dist-test-slave-sb2z"
server_key: "fdd3df2bbbe193992395d7b186621589"
server_key_iv: "853304f97783ff63b935068f9ddb0b97"
server_key_version: "encryptionkey@0"
I20260806 00:54:00.533710 32334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:00.550624 32334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:00.553954 32334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:00.554127 32334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:00.554744 32334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:00.555671 32334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:00.555723 32334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:00.555764 32334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:00.555816 32334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:00.564581 32334 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:36541
I20260806 00:54:00.564604 32496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:36541 every 8 connection(s)
I20260806 00:54:00.565475 32334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:54:00.573261 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 32334
I20260806 00:54:00.573396 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:54:00.573674 27959 external_mini_cluster.cc:1428] Setting key d7f9f50191cbb9b309bffd9bac483fa3
Aug 06 00:54:00 dist-test-slave-sb2z krb5kdc[32188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977640, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:00.580068 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:00.567401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:40701 (local address 127.27.77.254:42729)
0806 00:54:00.567715 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:00.567719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:00.568461 (+ 742us) server_negotiation.cc:407] Connection header received
0806 00:54:00.569906 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:00.569910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:00.569971 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:00.570080 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:00.571267 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:00.571892 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:00.573062 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:00.573309 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:00.573437 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:00.576268 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:00.576272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:00.576274 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:00.576298 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:00.577825 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:00.578439 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:00.578443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:00.578445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:00.578535 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:00.578889 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:00.578893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:00.578895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:00.579083 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:00.579191 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:00.579767 (+ 576us) server_negotiation.cc:299] Negotiation successful
0806 00:54:00.579900 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 00:54:00.580909 32500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:00.567812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:42729 (local address 127.27.77.193:40701)
0806 00:54:00.568293 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:00.568336 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:00.569685 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:00.570259 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:00.570265 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:00.570624 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:00.571106 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:00.571120 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:00.572031 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:00.572034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:00.572921 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:00.572931 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:00.573521 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:00.573527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:00.573697 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:00.574417 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:00.574441 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:00.576092 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:00.577993 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:00.577999 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:00.578025 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:00.578329 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:00.578631 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:00.578635 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:00.578637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:00.578790 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:00.579204 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:00.579212 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:00.579467 (+ 255us) client_negotiation.cc:769] Sending connection context
0806 00:54:00.579724 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 00:54:00.580073 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":116,"threads_started":1}
I20260806 00:54:00.582393 32497 heartbeater.cc:347] Connected to a master server at 127.27.77.254:42729
I20260806 00:54:00.582685 32497 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:00.583268 32497 heartbeater.cc:511] Master 127.27.77.254:42729 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
I20260806 00:54:00.585078 32251 ts_manager.cc:194] Registered new tserver with Master: a010cd2ef63f4a57b0dea6598af2b8fb (127.27.77.193:36541)
I20260806 00:54:00.586187 32251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:40701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:54:00.603607 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:42729
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:00.726651 32503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:00.726966 32503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:00.727018 32503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:00.731741 32503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:00.731814 32503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:00.731882 32503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:54:00.738372 32503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:42729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:00.739774 32503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:00.740957 32503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:00.751602 32511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:00.751578 32517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:00.751582 32510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:00.751945 32503 server_base.cc:1034] running on GCE node
I20260806 00:54:00.752364 32503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:00.753077 32503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:00.754287 32503 hybrid_clock.cc:648] HybridClock initialized: now 1785977640754264 us; error 34 us; skew 500 ppm
Aug 06 00:54:00 dist-test-slave-sb2z krb5kdc[32188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977640, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:00.757305 32503 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:54:00.758371 32503 webserver.cc:460] Webserver started at http://127.27.77.194:44269/ using document root <none> and password file <none>
I20260806 00:54:00.759011 32503 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:00.759117 32503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:00.759383 32503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:00.761868 32503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "e6882f8885754949b34dd2f261200907"
format_stamp: "Formatted at 2026-08-06 00:54:00 on dist-test-slave-sb2z"
server_key: "95385646dec4e16c87f3a7f3629b7b90"
server_key_iv: "d69dc405fb3d96117e829adcbef06159"
server_key_version: "encryptionkey@0"
I20260806 00:54:00.762519 32503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "e6882f8885754949b34dd2f261200907"
format_stamp: "Formatted at 2026-08-06 00:54:00 on dist-test-slave-sb2z"
server_key: "95385646dec4e16c87f3a7f3629b7b90"
server_key_iv: "d69dc405fb3d96117e829adcbef06159"
server_key_version: "encryptionkey@0"
I20260806 00:54:00.766710 32503 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 00:54:00.769315 32526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:00.770643 32503 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:54:00.770793 32503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "e6882f8885754949b34dd2f261200907"
format_stamp: "Formatted at 2026-08-06 00:54:00 on dist-test-slave-sb2z"
server_key: "95385646dec4e16c87f3a7f3629b7b90"
server_key_iv: "d69dc405fb3d96117e829adcbef06159"
server_key_version: "encryptionkey@0"
I20260806 00:54:00.770916 32503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:00.788980 32503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:00.792208 32503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:00.792424 32503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:00.793054 32503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:00.793913 32503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:00.793987 32503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:00.794059 32503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:00.794116 32503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:00.804302 32503 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:44671
I20260806 00:54:00.804323 32672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:44671 every 8 connection(s)
I20260806 00:54:00.805269 32503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:54:00.811815 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 32503
I20260806 00:54:00.811964 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:54:00.812306 27959 external_mini_cluster.cc:1428] Setting key bf127c6cf4eecb46add98dd948b151ba
Aug 06 00:54:00 dist-test-slave-sb2z krb5kdc[32188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977640, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:00.821106 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:00.807161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:48027 (local address 127.27.77.254:42729)
0806 00:54:00.807302 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:00.807306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:00.808311 (+ 1005us) server_negotiation.cc:407] Connection header received
0806 00:54:00.809769 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:00.809773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:00.809833 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:00.809944 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:00.811208 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:00.811821 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:00.813212 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:00.813447 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:00.813566 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:00.816698 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:00.816701 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:00.816703 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:00.816728 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:00.818514 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:00.819046 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:00.819052 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:00.819057 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:00.819170 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:00.819671 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:00.819676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:00.819680 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:00.819983 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:00.820097 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:00.820653 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 00:54:00.820814 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:54:00.821743 32676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:00.807637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:42729 (local address 127.27.77.194:48027)
0806 00:54:00.808139 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:00.808186 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:00.809538 (+ 1352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:00.810136 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:00.810143 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:00.810549 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:00.811044 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:00.811058 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:00.811960 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:00.811965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:00.813086 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:00.813097 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:00.813621 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:00.813626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:00.813841 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:00.814500 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:00.814527 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:00.816492 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:00.818653 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:00.818658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:00.818672 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:00.818928 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:00.819316 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:00.819319 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:00.819320 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:00.819467 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:00.820106 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:00.820112 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:00.820361 (+ 249us) client_negotiation.cc:769] Sending connection context
0806 00:54:00.820522 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 00:54:00.820811 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":115,"threads_started":1}
I20260806 00:54:00.823089 32673 heartbeater.cc:347] Connected to a master server at 127.27.77.254:42729
I20260806 00:54:00.823387 32673 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:00.824110 32673 heartbeater.cc:511] Master 127.27.77.254:42729 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
I20260806 00:54:00.825361 32251 ts_manager.cc:194] Registered new tserver with Master: e6882f8885754949b34dd2f261200907 (127.27.77.194:44671)
I20260806 00:54:00.826009 32251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:48027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:54:00.844959 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:42729
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:00.961450 32681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:00.961714 32681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:00.961755 32681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:00.965629 32681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:00.965684 32681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:00.965739 32681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:54:00.971498 32681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:42729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:00.972709 32681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:00.973776 32681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:00.984493 32691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:00.984520 32690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:00.984520 32694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:00.984719 32681 server_base.cc:1034] running on GCE node
I20260806 00:54:00.985255 32681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:00.985976 32681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:00.987159 32681 hybrid_clock.cc:648] HybridClock initialized: now 1785977640987142 us; error 31 us; skew 500 ppm
Aug 06 00:54:00 dist-test-slave-sb2z krb5kdc[32188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977640, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:00.990134 32681 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:54:00.991070 32681 webserver.cc:460] Webserver started at http://127.27.77.195:38777/ using document root <none> and password file <none>
I20260806 00:54:00.991729 32681 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:00.991827 32681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:00.992069 32681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:00.994153 32681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "ad39b463a9dd4c7e8344a694e184c21d"
format_stamp: "Formatted at 2026-08-06 00:54:00 on dist-test-slave-sb2z"
server_key: "2d065ed22d5b304bc7c38a44b7d18517"
server_key_iv: "02d77a700a2995eddf49c1f54ae6587f"
server_key_version: "encryptionkey@0"
I20260806 00:54:00.994745 32681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "ad39b463a9dd4c7e8344a694e184c21d"
format_stamp: "Formatted at 2026-08-06 00:54:00 on dist-test-slave-sb2z"
server_key: "2d065ed22d5b304bc7c38a44b7d18517"
server_key_iv: "02d77a700a2995eddf49c1f54ae6587f"
server_key_version: "encryptionkey@0"
I20260806 00:54:00.998672 32681 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 00:54:01.001686 32705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:01.003391 32681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:54:01.003578 32681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "ad39b463a9dd4c7e8344a694e184c21d"
format_stamp: "Formatted at 2026-08-06 00:54:00 on dist-test-slave-sb2z"
server_key: "2d065ed22d5b304bc7c38a44b7d18517"
server_key_iv: "02d77a700a2995eddf49c1f54ae6587f"
server_key_version: "encryptionkey@0"
I20260806 00:54:01.003731 32681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:01.019851 32681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:01.023445 32681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:01.023696 32681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:01.024381 32681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:01.025275 32681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:01.025349 32681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:01.025424 32681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:01.025476 32681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:01.035284 32681 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:42439
I20260806 00:54:01.035357 380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:42439 every 8 connection(s)
I20260806 00:54:01.036350 32681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:54:01.043241 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 32681
I20260806 00:54:01.043396 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:54:01.043767 27959 external_mini_cluster.cc:1428] Setting key 072c74f807711a61ede9a06e9dfbaf3d
Aug 06 00:54:01 dist-test-slave-sb2z krb5kdc[32188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977640, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:01.052008 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:01.038603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:59319 (local address 127.27.77.254:42729)
0806 00:54:01.038770 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:01.038774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:01.039683 (+ 909us) server_negotiation.cc:407] Connection header received
0806 00:54:01.041160 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:01.041163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:01.041229 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:01.041338 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:01.042760 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:01.043319 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:01.044870 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:01.045103 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:01.045221 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:01.048333 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:01.048337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:01.048340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:01.048376 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:01.049770 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:01.050326 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:01.050330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:01.050332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:01.050426 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:01.050799 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:01.050802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:01.050804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:01.051000 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:01.051120 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:01.051688 (+ 568us) server_negotiation.cc:299] Negotiation successful
0806 00:54:01.051824 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:54:01.052820 386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:01.038962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:42729 (local address 127.27.77.195:59319)
0806 00:54:01.039442 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:01.039488 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:01.040871 (+ 1383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:01.041528 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:01.041535 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:01.042011 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:01.042602 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:01.042618 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:01.043455 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:01.043459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:01.044744 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:01.044757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:01.045262 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:01.045266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:01.045455 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:01.046196 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:01.046221 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:01.048135 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:01.049898 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:01.049905 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:01.049920 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:01.050210 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:01.050534 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:01.050537 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:01.050539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:01.050691 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:01.051152 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:01.051158 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:01.051406 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 00:54:01.051631 (+ 225us) client_negotiation.cc:240] Negotiation successful
0806 00:54:01.051967 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":118,"threads_started":1}
I20260806 00:54:01.053992 384 heartbeater.cc:347] Connected to a master server at 127.27.77.254:42729
I20260806 00:54:01.054363 384 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:01.054908 384 heartbeater.cc:511] Master 127.27.77.254:42729 requested a full tablet report, sending...
I20260806 00:54:01.056205 32251 ts_manager.cc:194] Registered new tserver with Master: ad39b463a9dd4c7e8344a694e184c21d (127.27.77.195:42439)
I20260806 00:54:01.056890 32251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:59319
I20260806 00:54:01.058135 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 00:54:01 dist-test-slave-sb2z krb5kdc[32188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977641, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 00:54:01 dist-test-slave-sb2z krb5kdc[32188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977641, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:01.087299 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:01.078097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39004 (local address 127.27.77.254:42729)
0806 00:54:01.078276 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:01.078282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:01.078434 (+ 152us) server_negotiation.cc:407] Connection header received
0806 00:54:01.078636 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:01.078639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:01.078684 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:01.078774 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:01.079593 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:01.080144 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:01.081027 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:01.081264 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:01.081383 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:01.084080 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:01.084083 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:01.084085 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:01.084111 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:01.085486 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:01.085956 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:01.085964 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:01.085966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:01.086047 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:01.086345 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:01.086349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:01.086351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:01.086593 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:01.086754 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:01.086996 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 00:54:01.087120 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 00:54:01.087821 32251 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39004
I20260806 00:54:01.088974 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 32334
I20260806 00:54:01.101056 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 32503
I20260806 00:54:01.124552 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 32681
I20260806 00:54:01.135869 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 32208
2026-08-06T00:54:01Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2398 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:54:01 dist-test-slave-sb2z krb5kdc[398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:54:01 dist-test-slave-sb2z krb5kdc[398](info): set up 2 sockets
Aug 06 00:54:01 dist-test-slave-sb2z krb5kdc[398](info): commencing operation
krb5kdc: starting...
W20260806 00:54:02.409780 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:54:02 dist-test-slave-sb2z krb5kdc[398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:54:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:54:02.552328 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:40175
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:40175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:02.672605 416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:02.672950 416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:02.673038 416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:02.677256 416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:02.677352 416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:02.677389 416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:02.677426 416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:02.677467 416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:54:02.683003 416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:40175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:40175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:02.684248 416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:02.685352 416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:02.695612 424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:02.695672 428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:02.695611 425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:02.695837 416 server_base.cc:1034] running on GCE node
I20260806 00:54:02.696481 416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:02.697744 416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:02.698956 416 hybrid_clock.cc:648] HybridClock initialized: now 1785977642698934 us; error 35 us; skew 500 ppm
Aug 06 00:54:02 dist-test-slave-sb2z krb5kdc[398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977642, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:02.702172 416 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:54:02.703282 416 webserver.cc:460] Webserver started at http://127.27.77.254:41973/ using document root <none> and password file <none>
I20260806 00:54:02.703958 416 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:02.704043 416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:02.704236 416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:02.706401 416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "17f1893e921d4bfcb760e5638cfc47d1"
format_stamp: "Formatted at 2026-08-06 00:54:02 on dist-test-slave-sb2z"
server_key: "8ee0f2af311dbf522ef64196d6959e64"
server_key_iv: "e3eb74af8b765cf796cfe55eb88e9c60"
server_key_version: "encryptionkey@0"
I20260806 00:54:02.706967 416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "17f1893e921d4bfcb760e5638cfc47d1"
format_stamp: "Formatted at 2026-08-06 00:54:02 on dist-test-slave-sb2z"
server_key: "8ee0f2af311dbf522ef64196d6959e64"
server_key_iv: "e3eb74af8b765cf796cfe55eb88e9c60"
server_key_version: "encryptionkey@0"
I20260806 00:54:02.711227 416 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:54:02.714178 438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:02.715760 416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:54:02.715886 416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "17f1893e921d4bfcb760e5638cfc47d1"
format_stamp: "Formatted at 2026-08-06 00:54:02 on dist-test-slave-sb2z"
server_key: "8ee0f2af311dbf522ef64196d6959e64"
server_key_iv: "e3eb74af8b765cf796cfe55eb88e9c60"
server_key_version: "encryptionkey@0"
I20260806 00:54:02.715992 416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:02.736403 416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:02.739982 416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:02.740175 416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:02.748224 416 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:40175
I20260806 00:54:02.748241 514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:40175 every 8 connection(s)
I20260806 00:54:02.749279 416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:02.750912 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 416
I20260806 00:54:02.751057 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:54:02.751380 27959 external_mini_cluster.cc:1428] Setting key a4cad8851b37957804dc6bbcfcbfb44e
I20260806 00:54:02.752846 515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:02.759012 515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1: Bootstrap starting.
Aug 06 00:54:02 dist-test-slave-sb2z krb5kdc[398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:02.761713 515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:02.762766 515 log.cc:824] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:02.765512 515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1: No bootstrap required, opened a new log
I20260806 00:54:02.766934 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:02.753099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49240 (local address 127.27.77.254:40175)
0806 00:54:02.753634 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:02.753651 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:02.753690 (+ 39us) server_negotiation.cc:407] Connection header received
0806 00:54:02.754358 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:02.754377 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:02.754728 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:02.755153 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:02.756107 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:02.757130 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:02.758028 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:02.758524 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:02.758732 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:02.761548 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:02.761559 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:02.761579 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:02.761625 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:02.763667 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:02.764134 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:02.764140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:02.764147 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:02.764287 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:02.764559 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:02.764562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:02.764564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:02.765092 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:02.765299 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:02.765625 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 00:54:02.765950 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":159,"threads_started":1}
I20260806 00:54:02.770009 515 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f1893e921d4bfcb760e5638cfc47d1" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40175 } }
I20260806 00:54:02.770243 515 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:02.770279 515 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f1893e921d4bfcb760e5638cfc47d1, State: Initialized, Role: FOLLOWER
I20260806 00:54:02.770648 515 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f1893e921d4bfcb760e5638cfc47d1" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40175 } }
I20260806 00:54:02.770764 515 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:02.770807 515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:02.770872 515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:02.771921 515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f1893e921d4bfcb760e5638cfc47d1" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40175 } }
I20260806 00:54:02.772254 515 leader_election.cc:302] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17f1893e921d4bfcb760e5638cfc47d1; no voters:
I20260806 00:54:02.772535 515 leader_election.cc:288] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:54:02.772759 523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:02.773025 523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [term 1 LEADER]: Becoming Leader. State: Replica: 17f1893e921d4bfcb760e5638cfc47d1, State: Running, Role: LEADER
I20260806 00:54:02.773350 523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f1893e921d4bfcb760e5638cfc47d1" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40175 } }
I20260806 00:54:02.773919 515 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:02.775305 525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17f1893e921d4bfcb760e5638cfc47d1. Latest consensus state: current_term: 1 leader_uuid: "17f1893e921d4bfcb760e5638cfc47d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f1893e921d4bfcb760e5638cfc47d1" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40175 } } }
I20260806 00:54:02.775344 524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17f1893e921d4bfcb760e5638cfc47d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f1893e921d4bfcb760e5638cfc47d1" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40175 } } }
I20260806 00:54:02.775442 525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:02.775450 524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:02.775867 534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:02.779352 534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:02.785795 534 catalog_manager.cc:1347] Generated new cluster ID: 0212e7a4c1d54ed6b3b4f86c3938c6aa
I20260806 00:54:02.785914 534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:02.796939 534 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:54:02.798316 534 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:02.805711 534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17f1893e921d4bfcb760e5638cfc47d1: Generated new TSK 0
I20260806 00:54:02.806519 534 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:54:02.848994 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:40175
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:02.992559 551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:02.992888 551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:02.992964 551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:02.997480 551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:02.997593 551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:02.997713 551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:54:03.003445 551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:40175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:03.004763 551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:03.005854 551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:03.016109 565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:03.016176 551 server_base.cc:1034] running on GCE node
W20260806 00:54:03.016330 562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:03.016220 561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:03.016819 551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:03.017520 551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:03.018711 551 hybrid_clock.cc:648] HybridClock initialized: now 1785977643018676 us; error 49 us; skew 500 ppm
Aug 06 00:54:03 dist-test-slave-sb2z krb5kdc[398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977643, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:03.021975 551 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:54:03.022989 551 webserver.cc:460] Webserver started at http://127.27.77.193:34903/ using document root <none> and password file <none>
I20260806 00:54:03.023658 551 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:03.023753 551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:03.023993 551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:03.026127 551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "65a5a2ca03dc4745b67a8fc5b667e6fb"
format_stamp: "Formatted at 2026-08-06 00:54:03 on dist-test-slave-sb2z"
server_key: "1c609204403750c4c5747002388f8182"
server_key_iv: "997e448b76dfeb62eb33c9490d60eac9"
server_key_version: "encryptionkey@0"
I20260806 00:54:03.026721 551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "65a5a2ca03dc4745b67a8fc5b667e6fb"
format_stamp: "Formatted at 2026-08-06 00:54:03 on dist-test-slave-sb2z"
server_key: "1c609204403750c4c5747002388f8182"
server_key_iv: "997e448b76dfeb62eb33c9490d60eac9"
server_key_version: "encryptionkey@0"
I20260806 00:54:03.031068 551 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:54:03.033792 575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:03.035117 551 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:54:03.035272 551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "65a5a2ca03dc4745b67a8fc5b667e6fb"
format_stamp: "Formatted at 2026-08-06 00:54:03 on dist-test-slave-sb2z"
server_key: "1c609204403750c4c5747002388f8182"
server_key_iv: "997e448b76dfeb62eb33c9490d60eac9"
server_key_version: "encryptionkey@0"
I20260806 00:54:03.035403 551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:03.046417 551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:03.049906 551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:03.050151 551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:03.050798 551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:03.051762 551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:03.051832 551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:03.051903 551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:03.051945 551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:03.061712 551 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:38081
I20260806 00:54:03.061770 727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:38081 every 8 connection(s)
I20260806 00:54:03.062743 551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:54:03.069062 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 551
I20260806 00:54:03.069221 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:54:03.069510 27959 external_mini_cluster.cc:1428] Setting key 364ab82e6a1d7aeeef5e5a2812a5aba8
Aug 06 00:54:03 dist-test-slave-sb2z krb5kdc[398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977643, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:03.079157 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:03.064890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:44585 (local address 127.27.77.254:40175)
0806 00:54:03.065061 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:03.065067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:03.066060 (+ 993us) server_negotiation.cc:407] Connection header received
0806 00:54:03.067459 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:03.067462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:03.067517 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:03.067643 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:03.068953 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.069621 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.071129 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.071411 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.071506 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:03.075249 (+ 3743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:03.075253 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:03.075255 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:03.075281 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:03.076852 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:03.077475 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:03.077484 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:03.077485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:03.077566 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:03.077952 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:03.077956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:03.077958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:03.078166 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:03.078281 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:03.078832 (+ 551us) server_negotiation.cc:299] Negotiation successful
0806 00:54:03.078977 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:54:03.080163 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:03.065362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:40175 (local address 127.27.77.193:44585)
0806 00:54:03.065872 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:03.065917 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:03.067212 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:03.067821 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:03.067828 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:03.068222 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:03.068709 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:03.068722 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.069773 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.069777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:03.070957 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:03.070969 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.071663 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.071669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:03.071941 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:03.072695 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:03.072726 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:03.075034 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:03.076993 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:03.077000 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:03.077018 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:03.077349 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:03.077675 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:03.077679 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:03.077681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:03.077844 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:03.078280 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:03.078288 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:03.078574 (+ 286us) client_negotiation.cc:769] Sending connection context
0806 00:54:03.078788 (+ 214us) client_negotiation.cc:240] Negotiation successful
0806 00:54:03.079169 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":135,"threads_started":1}
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
I20260806 00:54:03.081642 728 heartbeater.cc:347] Connected to a master server at 127.27.77.254:40175
I20260806 00:54:03.081904 728 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:03.082490 728 heartbeater.cc:511] Master 127.27.77.254:40175 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
I20260806 00:54:03.084435 462 ts_manager.cc:194] Registered new tserver with Master: 65a5a2ca03dc4745b67a8fc5b667e6fb (127.27.77.193:38081)
I20260806 00:54:03.085609 462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:44585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:54:03.103463 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:40175
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:03.223057 736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:03.223346 736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:03.223425 736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:03.227730 736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:03.227797 736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:03.227895 736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:54:03.233630 736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:40175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:03.234793 736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:03.236070 736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:03.246608 743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:03.246614 745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:03.246629 742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:03.247155 736 server_base.cc:1034] running on GCE node
I20260806 00:54:03.247671 736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:03.248363 736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:03.249533 736 hybrid_clock.cc:648] HybridClock initialized: now 1785977643249517 us; error 29 us; skew 500 ppm
Aug 06 00:54:03 dist-test-slave-sb2z krb5kdc[398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977643, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:03.252578 736 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:54:03.253439 736 webserver.cc:460] Webserver started at http://127.27.77.194:41653/ using document root <none> and password file <none>
I20260806 00:54:03.254030 736 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:03.254099 736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:03.254273 736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:03.256417 736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "c01cac0f41fb4c47a9fe3dcbae7ae200"
format_stamp: "Formatted at 2026-08-06 00:54:03 on dist-test-slave-sb2z"
server_key: "acbd1512af0274aa06fdb3c5c4bfe879"
server_key_iv: "8a53e4426d37c4f5472abb9c1f2b382f"
server_key_version: "encryptionkey@0"
I20260806 00:54:03.256942 736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "c01cac0f41fb4c47a9fe3dcbae7ae200"
format_stamp: "Formatted at 2026-08-06 00:54:03 on dist-test-slave-sb2z"
server_key: "acbd1512af0274aa06fdb3c5c4bfe879"
server_key_iv: "8a53e4426d37c4f5472abb9c1f2b382f"
server_key_version: "encryptionkey@0"
I20260806 00:54:03.260819 736 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:54:03.263398 753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:03.264819 736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:54:03.264956 736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "c01cac0f41fb4c47a9fe3dcbae7ae200"
format_stamp: "Formatted at 2026-08-06 00:54:03 on dist-test-slave-sb2z"
server_key: "acbd1512af0274aa06fdb3c5c4bfe879"
server_key_iv: "8a53e4426d37c4f5472abb9c1f2b382f"
server_key_version: "encryptionkey@0"
I20260806 00:54:03.265074 736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:03.283006 736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:03.286449 736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:03.286679 736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:03.287319 736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:03.288228 736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:03.288303 736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:03.288379 736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:03.288431 736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:03.297837 736 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:37267
I20260806 00:54:03.297856 890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:37267 every 8 connection(s)
I20260806 00:54:03.298763 736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:54:03.301810 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 736
I20260806 00:54:03.301926 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:54:03.302202 27959 external_mini_cluster.cc:1428] Setting key 86973f3885285e802cd799efee95c253
Aug 06 00:54:03 dist-test-slave-sb2z krb5kdc[398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977643, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
I20260806 00:54:03.314981 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:03.300736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:36569 (local address 127.27.77.254:40175)
0806 00:54:03.300880 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:03.300884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:03.301804 (+ 920us) server_negotiation.cc:407] Connection header received
0806 00:54:03.303693 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:03.303697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:03.303755 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:03.303847 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:03.305245 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.305867 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.307761 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.308063 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.308198 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:03.310890 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:03.310897 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:03.310902 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:03.310939 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:03.312696 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:03.313255 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:03.313260 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:03.313263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:03.313365 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:03.313741 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:03.313744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:03.313746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:03.313974 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:03.314079 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:03.314731 (+ 652us) server_negotiation.cc:299] Negotiation successful
0806 00:54:03.314845 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:54:03.316035 893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:03.301120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:40175 (local address 127.27.77.194:36569)
0806 00:54:03.301619 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:03.301665 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:03.303390 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:03.304040 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:03.304050 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:03.304462 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:03.305089 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:03.305108 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.306027 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.306033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:03.307634 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:03.307648 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.308234 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.308237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:03.308391 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:03.309077 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:03.309096 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:03.310700 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:03.312827 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:03.312832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:03.312845 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:03.313139 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:03.313481 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:03.313483 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:03.313485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:03.313623 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:03.314112 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:03.314118 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:03.314464 (+ 346us) client_negotiation.cc:769] Sending connection context
0806 00:54:03.314659 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 00:54:03.314954 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
I20260806 00:54:03.317747 891 heartbeater.cc:347] Connected to a master server at 127.27.77.254:40175
I20260806 00:54:03.318046 891 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:03.318665 891 heartbeater.cc:511] Master 127.27.77.254:40175 requested a full tablet report, sending...
I20260806 00:54:03.320052 462 ts_manager.cc:194] Registered new tserver with Master: c01cac0f41fb4c47a9fe3dcbae7ae200 (127.27.77.194:37267)
I20260806 00:54:03.320983 462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:36569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:54:03.339481 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:40175
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:03.458977 896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:03.459286 896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:03.459334 896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:03.463594 896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:03.463698 896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:03.463765 896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:54:03.469440 896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:40175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:03.470567 896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:03.471604 896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:03.481927 908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:03.481942 896 server_base.cc:1034] running on GCE node
W20260806 00:54:03.482142 905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:03.482223 906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:03.482752 896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:03.483433 896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:03.484668 896 hybrid_clock.cc:648] HybridClock initialized: now 1785977643484638 us; error 43 us; skew 500 ppm
Aug 06 00:54:03 dist-test-slave-sb2z krb5kdc[398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977643, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:03.487769 896 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:54:03.488667 896 webserver.cc:460] Webserver started at http://127.27.77.195:44887/ using document root <none> and password file <none>
I20260806 00:54:03.489205 896 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:03.489264 896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:03.489435 896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:03.491484 896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "6f6b3d7439e941eab4af494ed73390b7"
format_stamp: "Formatted at 2026-08-06 00:54:03 on dist-test-slave-sb2z"
server_key: "20ad71c0c158daa48f1ab35bb366b29c"
server_key_iv: "816c60b627c8151449bd51f84011faf4"
server_key_version: "encryptionkey@0"
I20260806 00:54:03.492113 896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "6f6b3d7439e941eab4af494ed73390b7"
format_stamp: "Formatted at 2026-08-06 00:54:03 on dist-test-slave-sb2z"
server_key: "20ad71c0c158daa48f1ab35bb366b29c"
server_key_iv: "816c60b627c8151449bd51f84011faf4"
server_key_version: "encryptionkey@0"
I20260806 00:54:03.496168 896 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:54:03.498847 916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:03.500269 896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:54:03.500381 896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "6f6b3d7439e941eab4af494ed73390b7"
format_stamp: "Formatted at 2026-08-06 00:54:03 on dist-test-slave-sb2z"
server_key: "20ad71c0c158daa48f1ab35bb366b29c"
server_key_iv: "816c60b627c8151449bd51f84011faf4"
server_key_version: "encryptionkey@0"
I20260806 00:54:03.500535 896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:03.525429 896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:03.529009 896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:03.529247 896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:03.529911 896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:03.530861 896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:03.530910 896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:03.530977 896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:03.531013 896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:03.541049 896 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:44681
I20260806 00:54:03.541105 1049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:44681 every 8 connection(s)
I20260806 00:54:03.542006 896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:54:03.547744 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 896
I20260806 00:54:03.547891 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:54:03.548314 27959 external_mini_cluster.cc:1428] Setting key 0a875beaeb72f08ea5309971994c98b6
Aug 06 00:54:03 dist-test-slave-sb2z krb5kdc[398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977643, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:03.557361 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:03.543953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:37435 (local address 127.27.77.254:40175)
0806 00:54:03.544120 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:03.544125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:03.545139 (+ 1014us) server_negotiation.cc:407] Connection header received
0806 00:54:03.546565 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:03.546568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:03.546622 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:03.546714 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:03.548105 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.548876 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.550482 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.550773 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.550857 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:03.553742 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:03.553745 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:03.553747 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:03.553773 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:03.555197 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:03.555860 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:03.555862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:03.555863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:03.555943 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:03.556288 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:03.556291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:03.556292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:03.556473 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:03.556549 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:03.557126 (+ 577us) server_negotiation.cc:299] Negotiation successful
0806 00:54:03.557236 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 00:54:03.558218 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:03.544440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:40175 (local address 127.27.77.195:37435)
0806 00:54:03.544970 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:03.545016 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:03.546308 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:03.546885 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:03.546892 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:03.547257 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:03.547906 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:03.547941 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.549045 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.549048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:03.550258 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:03.550270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.550948 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.550951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:03.551120 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:03.551904 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:03.551923 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:03.553567 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:03.555333 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:03.555339 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:03.555356 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:03.555767 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:03.556035 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:03.556037 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:03.556039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:03.556193 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:03.556600 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:03.556606 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:03.556860 (+ 254us) client_negotiation.cc:769] Sending connection context
0806 00:54:03.557045 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 00:54:03.557328 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":125,"threads_started":1}
I20260806 00:54:03.559455 1050 heartbeater.cc:347] Connected to a master server at 127.27.77.254:40175
I20260806 00:54:03.559782 1050 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:03.560307 1050 heartbeater.cc:511] Master 127.27.77.254:40175 requested a full tablet report, sending...
I20260806 00:54:03.561637 460 ts_manager.cc:194] Registered new tserver with Master: 6f6b3d7439e941eab4af494ed73390b7 (127.27.77.195:44681)
I20260806 00:54:03.562392 460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:37435
I20260806 00:54:03.562668 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 00:54:03 dist-test-slave-sb2z krb5kdc[398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 00:54:03 dist-test-slave-sb2z krb5kdc[398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.193@KRBTEST.COM
I20260806 00:54:03.590387 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:03.581019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46096 (local address 127.27.77.193:38081)
0806 00:54:03.581378 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:03.581385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:03.581420 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:54:03.581471 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:03.581476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:03.581672 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:03.581793 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:03.582466 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.583234 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.584213 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.584608 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.584704 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:03.587077 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:03.587086 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:03.587130 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:03.587157 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:03.588774 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:03.589194 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:03.589199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:03.589201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:03.589283 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:03.589518 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:03.589520 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:03.589521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:03.589773 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:03.589932 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:03.590114 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 00:54:03.590243 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":159,"threads_started":1}
Aug 06 00:54:03 dist-test-slave-sb2z krb5kdc[398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:03.602649 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:03.593626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49250 (local address 127.27.77.254:40175)
0806 00:54:03.593794 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:03.593800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:03.593816 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:54:03.593951 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:03.593954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:03.594006 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:03.594103 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:03.594770 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.595337 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.596301 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:03.596695 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:03.596862 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:03.599332 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:03.599335 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:03.599337 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:03.599363 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:03.600869 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:03.601327 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:03.601331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:03.601332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:03.601409 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:03.601699 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:03.601703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:03.601705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:03.601941 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:03.602100 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:03.602346 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 00:54:03.602475 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 00:54:03.603089 460 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49250
I20260806 00:54:03.603817 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 551
I20260806 00:54:03.620204 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 736
I20260806 00:54:03.629302 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 896
I20260806 00:54:03.641067 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 416
2026-08-06T00:54:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2503 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:54:03 dist-test-slave-sb2z krb5kdc[1067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:54:03 dist-test-slave-sb2z krb5kdc[1067](info): set up 2 sockets
Aug 06 00:54:03 dist-test-slave-sb2z krb5kdc[1067](info): commencing operation
krb5kdc: starting...
W20260806 00:54:04.868224 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:54:04 dist-test-slave-sb2z krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:54:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:54:05.011422 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:43639
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:44279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:43639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:05.130784 1084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:05.131177 1084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:05.131295 1084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:05.135700 1084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:05.135805 1084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:05.135840 1084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:05.135880 1084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:05.135929 1084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:54:05.141690 1084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:44279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:43639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:43639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:05.142913 1084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:05.144040 1084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:05.153551 1089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:05.153653 1090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:05.153601 1092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:05.153772 1084 server_base.cc:1034] running on GCE node
I20260806 00:54:05.154629 1084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:05.155936 1084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:05.157150 1084 hybrid_clock.cc:648] HybridClock initialized: now 1785977645157117 us; error 48 us; skew 500 ppm
Aug 06 00:54:05 dist-test-slave-sb2z krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977645, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:05.160440 1084 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:54:05.161274 1084 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:05.161343 1084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:05.161590 1084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:05.163858 1084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "a95ed172966b4a0cbf4ed26d76e47422"
format_stamp: "Formatted at 2026-08-06 00:54:05 on dist-test-slave-sb2z"
server_key: "15cedc971c0ab12cc32f73d617dc2e05"
server_key_iv: "e20eea5d1b5f5a54de6366dc8fb94f3e"
server_key_version: "encryptionkey@0"
I20260806 00:54:05.164459 1084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "a95ed172966b4a0cbf4ed26d76e47422"
format_stamp: "Formatted at 2026-08-06 00:54:05 on dist-test-slave-sb2z"
server_key: "15cedc971c0ab12cc32f73d617dc2e05"
server_key_iv: "e20eea5d1b5f5a54de6366dc8fb94f3e"
server_key_version: "encryptionkey@0"
I20260806 00:54:05.168610 1084 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:54:05.171581 1100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:05.173079 1084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:54:05.173254 1084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "a95ed172966b4a0cbf4ed26d76e47422"
format_stamp: "Formatted at 2026-08-06 00:54:05 on dist-test-slave-sb2z"
server_key: "15cedc971c0ab12cc32f73d617dc2e05"
server_key_iv: "e20eea5d1b5f5a54de6366dc8fb94f3e"
server_key_version: "encryptionkey@0"
I20260806 00:54:05.173393 1084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:05.190752 1084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:05.194989 1084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:05.195263 1084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:05.206517 1084 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:43639
I20260806 00:54:05.206875 1173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:43639 every 8 connection(s)
I20260806 00:54:05.207576 1084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:05.210595 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 1084
I20260806 00:54:05.210747 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:54:05.211071 27959 external_mini_cluster.cc:1428] Setting key 3fe4f6bd36209b06e90559fc3df6042f
I20260806 00:54:05.211707 1174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:05.221360 1174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422: Bootstrap starting.
Aug 06 00:54:05 dist-test-slave-sb2z krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:05.223978 1174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:05.224925 1174 log.cc:824] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:05.227715 1174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422: No bootstrap required, opened a new log
I20260806 00:54:05.230374 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:05.212803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51662 (local address 127.27.77.254:43639)
0806 00:54:05.213404 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:05.213425 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:05.213477 (+ 52us) server_negotiation.cc:407] Connection header received
0806 00:54:05.214541 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:05.214564 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:05.214993 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:05.215516 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:05.216458 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:05.218405 (+ 1947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:05.219358 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:05.220157 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:05.220318 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:05.224073 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:05.224079 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:05.224120 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:05.224161 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:05.226093 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:05.226633 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:05.226639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:05.226645 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:05.226820 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:05.227216 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:05.227220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:05.227221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:05.227950 (+ 729us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:05.228178 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:05.228539 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 00:54:05.228910 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":176,"threads_started":1}
I20260806 00:54:05.232725 1174 raft_consensus.cc:348] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95ed172966b4a0cbf4ed26d76e47422" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43639 } }
I20260806 00:54:05.233175 1174 raft_consensus.cc:374] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:05.233346 1174 raft_consensus.cc:729] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a95ed172966b4a0cbf4ed26d76e47422, State: Initialized, Role: FOLLOWER
I20260806 00:54:05.233995 1174 consensus_queue.cc:260] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95ed172966b4a0cbf4ed26d76e47422" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43639 } }
I20260806 00:54:05.234194 1174 raft_consensus.cc:388] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:05.234289 1174 raft_consensus.cc:482] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:05.234421 1174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:05.236075 1174 raft_consensus.cc:504] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95ed172966b4a0cbf4ed26d76e47422" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43639 } }
I20260806 00:54:05.236552 1174 leader_election.cc:302] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a95ed172966b4a0cbf4ed26d76e47422; no voters:
I20260806 00:54:05.236979 1174 leader_election.cc:288] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:54:05.237203 1181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:05.237531 1181 raft_consensus.cc:686] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [term 1 LEADER]: Becoming Leader. State: Replica: a95ed172966b4a0cbf4ed26d76e47422, State: Running, Role: LEADER
I20260806 00:54:05.237910 1181 consensus_queue.cc:237] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95ed172966b4a0cbf4ed26d76e47422" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43639 } }
I20260806 00:54:05.238607 1174 sys_catalog.cc:564] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:05.239317 1181 sys_catalog.cc:455] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a95ed172966b4a0cbf4ed26d76e47422. Latest consensus state: current_term: 1 leader_uuid: "a95ed172966b4a0cbf4ed26d76e47422" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95ed172966b4a0cbf4ed26d76e47422" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43639 } } }
I20260806 00:54:05.239570 1181 sys_catalog.cc:458] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:05.239796 1182 sys_catalog.cc:455] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a95ed172966b4a0cbf4ed26d76e47422" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95ed172966b4a0cbf4ed26d76e47422" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43639 } } }
I20260806 00:54:05.239894 1182 sys_catalog.cc:458] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:05.240080 1189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:05.243441 1189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:05.251045 1189 catalog_manager.cc:1347] Generated new cluster ID: 00267b07d97a425ba072dda206e8ff02
I20260806 00:54:05.251164 1189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:05.278301 1189 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:54:05.279353 1189 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:05.293892 1189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422: Generated new TSK 0
I20260806 00:54:05.294735 1189 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:54:05.340034 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:43639
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:44279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:05.482817 1206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:05.483104 1206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:05.483156 1206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:05.487130 1206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:05.487191 1206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:05.487249 1206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:54:05.492517 1206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:44279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:43639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:05.493541 1206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:05.494602 1206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:54:05.504381 1206 server_base.cc:1034] running on GCE node
W20260806 00:54:05.504330 1211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:05.504375 1212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:05.504330 1214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:05.504987 1206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:05.505610 1206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:05.506785 1206 hybrid_clock.cc:648] HybridClock initialized: now 1785977645506770 us; error 28 us; skew 500 ppm
Aug 06 00:54:05 dist-test-slave-sb2z krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977645, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:05.509907 1206 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:54:05.510723 1206 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:05.510789 1206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:05.510960 1206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:05.513064 1206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "21a75dac4a874fecbd60d77ff66f8000"
format_stamp: "Formatted at 2026-08-06 00:54:05 on dist-test-slave-sb2z"
server_key: "5ec04f19aa73202741e45dc3dbdcbb1a"
server_key_iv: "84bc21dd76a980f908d86c33a9bb075a"
server_key_version: "encryptionkey@0"
I20260806 00:54:05.513623 1206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "21a75dac4a874fecbd60d77ff66f8000"
format_stamp: "Formatted at 2026-08-06 00:54:05 on dist-test-slave-sb2z"
server_key: "5ec04f19aa73202741e45dc3dbdcbb1a"
server_key_iv: "84bc21dd76a980f908d86c33a9bb075a"
server_key_version: "encryptionkey@0"
I20260806 00:54:05.517496 1206 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 00:54:05.520098 1223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:05.521387 1206 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:54:05.521488 1206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "21a75dac4a874fecbd60d77ff66f8000"
format_stamp: "Formatted at 2026-08-06 00:54:05 on dist-test-slave-sb2z"
server_key: "5ec04f19aa73202741e45dc3dbdcbb1a"
server_key_iv: "84bc21dd76a980f908d86c33a9bb075a"
server_key_version: "encryptionkey@0"
I20260806 00:54:05.521580 1206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:05.551084 1206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:05.554520 1206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:05.554703 1206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:05.555280 1206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:05.556172 1206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:05.556227 1206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:05.556269 1206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:05.556285 1206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:05.565994 1206 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:42859
I20260806 00:54:05.566044 1367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:42859 every 8 connection(s)
I20260806 00:54:05.566697 1206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:54:05.569690 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 1206
I20260806 00:54:05.569837 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:54:05.570163 27959 external_mini_cluster.cc:1428] Setting key 74ea653380590a0d6bce77e9f1f69130
Aug 06 00:54:05 dist-test-slave-sb2z krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977645, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
I20260806 00:54:05.583859 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:05.568662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:49367 (local address 127.27.77.254:43639)
0806 00:54:05.568850 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:05.568855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:05.569791 (+ 936us) server_negotiation.cc:407] Connection header received
0806 00:54:05.571663 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:05.571667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:05.571726 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:05.571814 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:05.573092 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:05.573944 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:05.575854 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:05.576088 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:05.576177 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:05.579458 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:05.579465 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:05.579469 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:05.579507 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:05.581473 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:05.582039 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:05.582046 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:05.582049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:05.582155 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:05.582511 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:05.582514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:05.582516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:05.582805 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:05.582930 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:05.583497 (+ 567us) server_negotiation.cc:299] Negotiation successful
0806 00:54:05.583731 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 00:54:05.584906 1372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:05.569094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43639 (local address 127.27.77.193:49367)
0806 00:54:05.569614 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:05.569659 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:05.571348 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:05.572032 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:05.572039 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:05.572427 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:05.572927 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:05.572941 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:05.574104 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:05.574110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:05.575729 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:05.575744 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:05.576273 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:05.576280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:05.576563 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:05.577268 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:05.577322 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:05.579257 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:05.581621 (+ 2364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:05.581627 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:05.581642 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:05.581918 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:05.582269 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:05.582271 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:05.582272 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:05.582412 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:05.582931 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:05.582937 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:05.583245 (+ 308us) client_negotiation.cc:769] Sending connection context
0806 00:54:05.583431 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 00:54:05.583886 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":122,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
I20260806 00:54:05.586504 1368 heartbeater.cc:347] Connected to a master server at 127.27.77.254:43639
I20260806 00:54:05.586800 1368 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:05.587334 1368 heartbeater.cc:511] Master 127.27.77.254:43639 requested a full tablet report, sending...
I20260806 00:54:05.589191 1123 ts_manager.cc:194] Registered new tserver with Master: 21a75dac4a874fecbd60d77ff66f8000 (127.27.77.193:42859)
I20260806 00:54:05.590387 1123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:49367
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:54:05.605114 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:43639
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:44279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:05.729140 1375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:05.729480 1375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:05.729557 1375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:05.733769 1375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:05.733834 1375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:05.733894 1375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:54:05.739636 1375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:44279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:43639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:05.741147 1375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:05.742228 1375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:05.752169 1384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:05.752178 1381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:05.752568 1375 server_base.cc:1034] running on GCE node
W20260806 00:54:05.752189 1382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:05.753260 1375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:05.754020 1375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:05.755249 1375 hybrid_clock.cc:648] HybridClock initialized: now 1785977645755215 us; error 51 us; skew 500 ppm
Aug 06 00:54:05 dist-test-slave-sb2z krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977645, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:05.758556 1375 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:54:05.759443 1375 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:05.759583 1375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:05.759859 1375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:05.762046 1375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "7501d7d926c949ccb4afd126d0dba3d9"
format_stamp: "Formatted at 2026-08-06 00:54:05 on dist-test-slave-sb2z"
server_key: "3e9699056cce7362226a78efba1a3c49"
server_key_iv: "7a0bf128136461e4744c535011cde1af"
server_key_version: "encryptionkey@0"
I20260806 00:54:05.762666 1375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "7501d7d926c949ccb4afd126d0dba3d9"
format_stamp: "Formatted at 2026-08-06 00:54:05 on dist-test-slave-sb2z"
server_key: "3e9699056cce7362226a78efba1a3c49"
server_key_iv: "7a0bf128136461e4744c535011cde1af"
server_key_version: "encryptionkey@0"
I20260806 00:54:05.766706 1375 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 00:54:05.769490 1390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:05.770856 1375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:54:05.771013 1375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "7501d7d926c949ccb4afd126d0dba3d9"
format_stamp: "Formatted at 2026-08-06 00:54:05 on dist-test-slave-sb2z"
server_key: "3e9699056cce7362226a78efba1a3c49"
server_key_iv: "7a0bf128136461e4744c535011cde1af"
server_key_version: "encryptionkey@0"
I20260806 00:54:05.771144 1375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:05.788717 1375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:05.792204 1375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:05.792434 1375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:05.793110 1375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:05.794087 1375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:05.794155 1375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:05.794227 1375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:05.794255 1375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:05.804348 1375 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:42251
I20260806 00:54:05.804369 1541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:42251 every 8 connection(s)
I20260806 00:54:05.805008 1375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:54:05.813758 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 1375
I20260806 00:54:05.813911 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:54:05.814222 27959 external_mini_cluster.cc:1428] Setting key 14bcb32f46e45948084052c590301663
Aug 06 00:54:05 dist-test-slave-sb2z krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977645, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:05.820290 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:05.806952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:39835 (local address 127.27.77.254:43639)
0806 00:54:05.807143 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:05.807147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:05.808144 (+ 997us) server_negotiation.cc:407] Connection header received
0806 00:54:05.809546 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:05.809550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:05.809613 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:05.809727 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:05.810989 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:05.811572 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:05.812752 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:05.812983 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:05.813126 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:05.816145 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:05.816149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:05.816151 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:05.816174 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:05.817717 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:05.818399 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:05.818405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:05.818408 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:05.818515 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:05.818886 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:05.818891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:05.818894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:05.819159 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:05.819280 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:05.819873 (+ 593us) server_negotiation.cc:299] Negotiation successful
0806 00:54:05.820027 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:54:05.821082 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:05.807372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43639 (local address 127.27.77.194:39835)
0806 00:54:05.807952 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:05.807999 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:05.809316 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:05.809916 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:05.809923 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:05.810310 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:05.810806 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:05.810820 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:05.811714 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:05.811723 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:05.812610 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:05.812621 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:05.813159 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:05.813164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:05.813353 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:05.814087 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:05.814106 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:05.815918 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:05.817847 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:05.817857 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:05.817878 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:05.818281 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:05.818624 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:05.818627 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:05.818628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:05.818788 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:05.819277 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:05.819283 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:05.819612 (+ 329us) client_negotiation.cc:769] Sending connection context
0806 00:54:05.819787 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 00:54:05.820072 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":213,"threads_started":1}
I20260806 00:54:05.822392 1542 heartbeater.cc:347] Connected to a master server at 127.27.77.254:43639
I20260806 00:54:05.822687 1542 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:05.823171 1542 heartbeater.cc:511] Master 127.27.77.254:43639 requested a full tablet report, sending...
I20260806 00:54:05.824394 1123 ts_manager.cc:194] Registered new tserver with Master: 7501d7d926c949ccb4afd126d0dba3d9 (127.27.77.194:42251)
I20260806 00:54:05.825166 1123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:39835
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:54:05.847189 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:43639
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:44279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:05.967590 1547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:05.967876 1547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:05.967963 1547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:05.972138 1547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:05.972203 1547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:05.972296 1547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:54:05.978135 1547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:44279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:43639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:05.979226 1547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:05.980333 1547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:05.990453 1554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:05.990535 1559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:05.990648 1556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:05.990751 1547 server_base.cc:1034] running on GCE node
I20260806 00:54:05.991291 1547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:05.992125 1547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:05.993355 1547 hybrid_clock.cc:648] HybridClock initialized: now 1785977645993332 us; error 37 us; skew 500 ppm
Aug 06 00:54:05 dist-test-slave-sb2z krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977645, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:05.996470 1547 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:54:05.997262 1547 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:05.997363 1547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:05.997628 1547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:05.999856 1547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "de2288b4d25c41e58dfbc3069a566e35"
format_stamp: "Formatted at 2026-08-06 00:54:05 on dist-test-slave-sb2z"
server_key: "b87737d50ece565a889efd478f343800"
server_key_iv: "5c352bb91d78a7d62ab45c169b55f176"
server_key_version: "encryptionkey@0"
I20260806 00:54:06.000430 1547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "de2288b4d25c41e58dfbc3069a566e35"
format_stamp: "Formatted at 2026-08-06 00:54:05 on dist-test-slave-sb2z"
server_key: "b87737d50ece565a889efd478f343800"
server_key_iv: "5c352bb91d78a7d62ab45c169b55f176"
server_key_version: "encryptionkey@0"
I20260806 00:54:06.004429 1547 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:54:06.007208 1567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:06.008859 1547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:54:06.009030 1547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "de2288b4d25c41e58dfbc3069a566e35"
format_stamp: "Formatted at 2026-08-06 00:54:05 on dist-test-slave-sb2z"
server_key: "b87737d50ece565a889efd478f343800"
server_key_iv: "5c352bb91d78a7d62ab45c169b55f176"
server_key_version: "encryptionkey@0"
I20260806 00:54:06.009172 1547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:06.051915 1547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:06.055632 1547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:06.055857 1547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:06.056521 1547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:06.057440 1547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:06.057531 1547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:06.057608 1547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:06.057662 1547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:06.067430 1547 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:42521
I20260806 00:54:06.067476 1714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:42521 every 8 connection(s)
I20260806 00:54:06.068218 1547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:54:06.076215 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 1547
I20260806 00:54:06.076368 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:54:06.076699 27959 external_mini_cluster.cc:1428] Setting key 925d1dff24e47c70a2b4d76da51e122a
Aug 06 00:54:06 dist-test-slave-sb2z krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977645, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:06.083923 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.070318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:34403 (local address 127.27.77.254:43639)
0806 00:54:06.070513 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:06.070517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:06.071468 (+ 951us) server_negotiation.cc:407] Connection header received
0806 00:54:06.072958 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:06.072961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:06.073024 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:06.073118 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:06.074358 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.075014 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.076388 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.076732 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.076854 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:06.080251 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:06.080254 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:06.080256 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:06.080283 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:06.081709 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:06.082296 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:06.082299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:06.082300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:06.082378 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:06.082710 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:06.082713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:06.082714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:06.082893 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:06.082968 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:06.083663 (+ 695us) server_negotiation.cc:299] Negotiation successful
0806 00:54:06.083789 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 00:54:06.084856 1718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.070753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43639 (local address 127.27.77.195:34403)
0806 00:54:06.071285 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:06.071333 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:06.072736 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:06.073301 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:06.073307 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:06.073682 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:06.074186 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:06.074200 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.075146 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.075150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:06.076223 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:06.076234 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.076989 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.076996 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:06.077238 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:06.078013 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:06.078037 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:06.080083 (+ 2046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:06.081847 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:06.081853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:06.081867 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:06.082191 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:06.082462 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:06.082467 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:06.082468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:06.082615 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:06.083012 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:06.083017 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:06.083301 (+ 284us) client_negotiation.cc:769] Sending connection context
0806 00:54:06.083466 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 00:54:06.083910 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":141,"threads_started":1}
I20260806 00:54:06.086179 1715 heartbeater.cc:347] Connected to a master server at 127.27.77.254:43639
I20260806 00:54:06.086445 1715 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:06.086922 1715 heartbeater.cc:511] Master 127.27.77.254:43639 requested a full tablet report, sending...
I20260806 00:54:06.088109 1123 ts_manager.cc:194] Registered new tserver with Master: de2288b4d25c41e58dfbc3069a566e35 (127.27.77.195:42521)
I20260806 00:54:06.088690 1123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:34403
I20260806 00:54:06.091238 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:54:06.100786 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.093387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51666 (local address 127.27.77.254:43639)
0806 00:54:06.093532 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:06.093536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:06.093746 (+ 210us) server_negotiation.cc:407] Connection header received
0806 00:54:06.093850 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:06.093852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:06.093889 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:06.093956 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:06.094776 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.095283 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.096149 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.096366 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.096447 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:06.097809 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:06.097811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:06.097813 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:06.097836 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:06.099143 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:06.099630 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:06.099633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:06.099634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:06.099711 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:06.099977 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:06.099979 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:06.099980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:06.100190 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:06.100296 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:06.100553 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 00:54:06.100660 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:54:06.103852 1123 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:54:06.106916 1123 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:54:06.122622 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.117509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52954 (local address 127.27.77.194:42251)
0806 00:54:06.117897 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:06.117911 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:06.117930 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:54:06.118006 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:06.118012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:06.118193 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:06.118350 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:06.118576 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.119380 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.121317 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.121903 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.122043 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:06.122204 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:06.122333 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 00:54:06.122432 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":127,"threads_started":1}
I20260806 00:54:06.122617 1735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.117068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:42251 (local address 127.0.0.1:52954)
0806 00:54:06.117544 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:06.117556 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:06.117653 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:06.118346 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:06.118351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:06.118374 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:06.118478 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:06.118485 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.119518 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.119546 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:06.121217 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:06.121227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.122041 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.122045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:06.122158 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:06.122196 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 00:54:06.122305 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 00:54:06.122426 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":229,"thread_start_us":93,"threads_started":1}
I20260806 00:54:06.122663 1731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.116301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:42859 (local address 127.0.0.1:59204)
0806 00:54:06.117188 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:06.117242 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:06.117390 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:06.118054 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:06.118060 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:06.118083 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:06.118238 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:06.118244 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.119729 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.119733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:06.121134 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:06.121146 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.122221 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.122225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:06.122311 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:06.122334 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:54:06.122388 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 00:54:06.122459 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":3,"thread_start_us":81,"threads_started":1}
I20260806 00:54:06.123036 1734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.116757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45758 (local address 127.27.77.195:42521)
0806 00:54:06.117208 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:06.117222 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:06.117888 (+ 666us) server_negotiation.cc:407] Connection header received
0806 00:54:06.118007 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:06.118012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:06.118193 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:06.118350 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:06.118805 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.120145 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.121225 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.122066 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.122197 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:06.122344 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:06.122715 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 00:54:06.122843 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":153,"threads_started":1}
I20260806 00:54:06.123606 1733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.116606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:42521 (local address 127.0.0.1:45758)
0806 00:54:06.117351 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:06.117365 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:06.117888 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:06.118532 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:06.118537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:06.118559 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:06.118663 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:06.118671 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.120308 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.120312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:06.121086 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:06.121097 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.122404 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.122407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:06.122491 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:06.122513 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:54:06.122579 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 00:54:06.122636 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":4,"thread_start_us":102,"threads_started":1}
I20260806 00:54:06.123611 1732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.116449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59204 (local address 127.27.77.193:42859)
0806 00:54:06.116879 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:06.116886 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:06.117523 (+ 637us) server_negotiation.cc:407] Connection header received
0806 00:54:06.117625 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:06.117633 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:06.117868 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:06.118071 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:06.118395 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.119592 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.121275 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.122076 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.123071 (+ 995us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:06.123308 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:06.123352 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 00:54:06.123425 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":114,"threads_started":1}
I20260806 00:54:06.126590 1282 tablet_service.cc:1467] Processing CreateTablet for tablet 9f3723caac5248249ea0305fb63b90f7 (DEFAULT_TABLE table=test-table [id=64e7e60bdf9d4b65b5698fa1299ddc99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:06.126611 1627 tablet_service.cc:1467] Processing CreateTablet for tablet 9f3723caac5248249ea0305fb63b90f7 (DEFAULT_TABLE table=test-table [id=64e7e60bdf9d4b65b5698fa1299ddc99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:06.126603 1454 tablet_service.cc:1467] Processing CreateTablet for tablet 9f3723caac5248249ea0305fb63b90f7 (DEFAULT_TABLE table=test-table [id=64e7e60bdf9d4b65b5698fa1299ddc99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:06.127725 1627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f3723caac5248249ea0305fb63b90f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:06.127722 1454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f3723caac5248249ea0305fb63b90f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:06.127722 1282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f3723caac5248249ea0305fb63b90f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:06.134298 1737 tablet_bootstrap.cc:492] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000: Bootstrap starting.
I20260806 00:54:06.135864 1739 tablet_bootstrap.cc:492] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35: Bootstrap starting.
I20260806 00:54:06.136813 1737 tablet_bootstrap.cc:654] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:06.137310 1738 tablet_bootstrap.cc:492] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9: Bootstrap starting.
I20260806 00:54:06.137823 1739 tablet_bootstrap.cc:654] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:06.137782 1737 log.cc:824] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:06.139348 1739 log.cc:824] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:06.139714 1738 tablet_bootstrap.cc:654] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:06.140225 1737 tablet_bootstrap.cc:492] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000: No bootstrap required, opened a new log
I20260806 00:54:06.140445 1737 ts_tablet_manager.cc:1397] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:54:06.141198 1738 log.cc:824] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:06.141860 1739 tablet_bootstrap.cc:492] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35: No bootstrap required, opened a new log
I20260806 00:54:06.142102 1739 ts_tablet_manager.cc:1397] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:54:06.143353 1738 tablet_bootstrap.cc:492] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9: No bootstrap required, opened a new log
I20260806 00:54:06.143569 1738 ts_tablet_manager.cc:1397] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.004s
I20260806 00:54:06.144593 1737 raft_consensus.cc:348] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a75dac4a874fecbd60d77ff66f8000" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42859 } } peers { permanent_uuid: "de2288b4d25c41e58dfbc3069a566e35" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42521 } } peers { permanent_uuid: "7501d7d926c949ccb4afd126d0dba3d9" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42251 } }
I20260806 00:54:06.144841 1737 raft_consensus.cc:374] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:06.144901 1737 raft_consensus.cc:729] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a75dac4a874fecbd60d77ff66f8000, State: Initialized, Role: FOLLOWER
I20260806 00:54:06.145427 1737 consensus_queue.cc:260] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a75dac4a874fecbd60d77ff66f8000" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42859 } } peers { permanent_uuid: "de2288b4d25c41e58dfbc3069a566e35" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42521 } } peers { permanent_uuid: "7501d7d926c949ccb4afd126d0dba3d9" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42251 } }
I20260806 00:54:06.146068 1739 raft_consensus.cc:348] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a75dac4a874fecbd60d77ff66f8000" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42859 } } peers { permanent_uuid: "de2288b4d25c41e58dfbc3069a566e35" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42521 } } peers { permanent_uuid: "7501d7d926c949ccb4afd126d0dba3d9" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42251 } }
I20260806 00:54:06.146359 1739 raft_consensus.cc:374] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:06.146435 1739 raft_consensus.cc:729] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2288b4d25c41e58dfbc3069a566e35, State: Initialized, Role: FOLLOWER
I20260806 00:54:06.146492 1737 ts_tablet_manager.cc:1428] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:54:06.146893 1368 heartbeater.cc:503] Master 127.27.77.254:43639 was elected leader, sending a full tablet report...
I20260806 00:54:06.146914 1739 consensus_queue.cc:260] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a75dac4a874fecbd60d77ff66f8000" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42859 } } peers { permanent_uuid: "de2288b4d25c41e58dfbc3069a566e35" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42521 } } peers { permanent_uuid: "7501d7d926c949ccb4afd126d0dba3d9" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42251 } }
I20260806 00:54:06.147707 1715 heartbeater.cc:503] Master 127.27.77.254:43639 was elected leader, sending a full tablet report...
I20260806 00:54:06.147518 1738 raft_consensus.cc:348] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a75dac4a874fecbd60d77ff66f8000" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42859 } } peers { permanent_uuid: "de2288b4d25c41e58dfbc3069a566e35" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42521 } } peers { permanent_uuid: "7501d7d926c949ccb4afd126d0dba3d9" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42251 } }
I20260806 00:54:06.147831 1738 raft_consensus.cc:374] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:06.147886 1738 raft_consensus.cc:729] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7501d7d926c949ccb4afd126d0dba3d9, State: Initialized, Role: FOLLOWER
I20260806 00:54:06.148030 1739 ts_tablet_manager.cc:1428] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:54:06.148307 1738 consensus_queue.cc:260] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a75dac4a874fecbd60d77ff66f8000" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42859 } } peers { permanent_uuid: "de2288b4d25c41e58dfbc3069a566e35" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42521 } } peers { permanent_uuid: "7501d7d926c949ccb4afd126d0dba3d9" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42251 } }
I20260806 00:54:06.149003 1542 heartbeater.cc:503] Master 127.27.77.254:43639 was elected leader, sending a full tablet report...
I20260806 00:54:06.149521 1738 ts_tablet_manager.cc:1428] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260806 00:54:06.254061 1746 raft_consensus.cc:482] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:54:06.254297 1746 raft_consensus.cc:504] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a75dac4a874fecbd60d77ff66f8000" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42859 } } peers { permanent_uuid: "de2288b4d25c41e58dfbc3069a566e35" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42521 } } peers { permanent_uuid: "7501d7d926c949ccb4afd126d0dba3d9" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42251 } }
I20260806 00:54:06.255568 1746 leader_election.cc:288] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21a75dac4a874fecbd60d77ff66f8000 (127.27.77.193:42859), de2288b4d25c41e58dfbc3069a566e35 (127.27.77.195:42521)
I20260806 00:54:06.261008 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.255826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:42859 (local address 127.27.77.194:37179)
0806 00:54:06.256070 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:06.256087 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:06.256244 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:06.256602 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:06.256606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:06.256625 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:06.256736 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:06.256741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.257842 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.257845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:06.259414 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:06.259424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.260410 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.260414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:06.260610 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:06.260637 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 00:54:06.260712 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 00:54:06.260789 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":150}
I20260806 00:54:06.261049 1734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.256076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:40113 (local address 127.27.77.195:42521)
0806 00:54:06.256231 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:06.256239 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:06.256520 (+ 281us) server_negotiation.cc:407] Connection header received
0806 00:54:06.256642 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:06.256644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:06.256698 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:06.256771 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:06.257056 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.257804 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.259588 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.260224 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.260370 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:06.260426 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:06.260747 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 00:54:06.260901 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 00:54:06.261055 1732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.255939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:37179 (local address 127.27.77.193:42859)
0806 00:54:06.256098 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:06.256103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:06.256197 (+ 94us) server_negotiation.cc:407] Connection header received
0806 00:54:06.256395 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:06.256398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:06.256449 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:06.256566 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:06.256889 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.257686 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.259588 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.260236 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.260370 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:06.260426 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:06.260858 (+ 432us) server_negotiation.cc:299] Negotiation successful
0806 00:54:06.260951 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 00:54:06.261070 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.255932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:42521 (local address 127.27.77.194:40113)
0806 00:54:06.256413 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:06.256424 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:06.256508 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:06.256808 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:06.256812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:06.256833 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:06.256942 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:06.256947 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.257914 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.257916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:06.259411 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:06.259424 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.260412 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.260415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:06.260561 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:06.260596 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 00:54:06.260697 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 00:54:06.260789 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":85,"thread_start_us":157,"threads_started":1}
I20260806 00:54:06.261920 1311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f3723caac5248249ea0305fb63b90f7" candidate_uuid: "7501d7d926c949ccb4afd126d0dba3d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a75dac4a874fecbd60d77ff66f8000" is_pre_election: true
I20260806 00:54:06.261946 1656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f3723caac5248249ea0305fb63b90f7" candidate_uuid: "7501d7d926c949ccb4afd126d0dba3d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de2288b4d25c41e58dfbc3069a566e35" is_pre_election: true
I20260806 00:54:06.262279 1311 raft_consensus.cc:2454] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7501d7d926c949ccb4afd126d0dba3d9 in term 0.
I20260806 00:54:06.262274 1656 raft_consensus.cc:2454] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7501d7d926c949ccb4afd126d0dba3d9 in term 0.
I20260806 00:54:06.262846 1394 leader_election.cc:302] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 21a75dac4a874fecbd60d77ff66f8000, 7501d7d926c949ccb4afd126d0dba3d9, de2288b4d25c41e58dfbc3069a566e35; no voters:
I20260806 00:54:06.263144 1746 raft_consensus.cc:2781] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:54:06.263252 1746 raft_consensus.cc:482] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:54:06.263355 1746 raft_consensus.cc:3037] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:06.264643 1746 raft_consensus.cc:504] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a75dac4a874fecbd60d77ff66f8000" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42859 } } peers { permanent_uuid: "de2288b4d25c41e58dfbc3069a566e35" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42521 } } peers { permanent_uuid: "7501d7d926c949ccb4afd126d0dba3d9" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42251 } }
I20260806 00:54:06.265103 1746 leader_election.cc:288] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [CANDIDATE]: Term 1 election: Requested vote from peers 21a75dac4a874fecbd60d77ff66f8000 (127.27.77.193:42859), de2288b4d25c41e58dfbc3069a566e35 (127.27.77.195:42521)
I20260806 00:54:06.265503 1311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f3723caac5248249ea0305fb63b90f7" candidate_uuid: "7501d7d926c949ccb4afd126d0dba3d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a75dac4a874fecbd60d77ff66f8000"
I20260806 00:54:06.265560 1656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f3723caac5248249ea0305fb63b90f7" candidate_uuid: "7501d7d926c949ccb4afd126d0dba3d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de2288b4d25c41e58dfbc3069a566e35"
I20260806 00:54:06.265676 1311 raft_consensus.cc:3037] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:06.265691 1656 raft_consensus.cc:3037] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:06.267508 1656 raft_consensus.cc:2454] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7501d7d926c949ccb4afd126d0dba3d9 in term 1.
I20260806 00:54:06.267508 1311 raft_consensus.cc:2454] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7501d7d926c949ccb4afd126d0dba3d9 in term 1.
I20260806 00:54:06.267956 1393 leader_election.cc:302] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a75dac4a874fecbd60d77ff66f8000, 7501d7d926c949ccb4afd126d0dba3d9; no voters:
I20260806 00:54:06.268286 1746 raft_consensus.cc:2781] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:06.268539 1746 raft_consensus.cc:686] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [term 1 LEADER]: Becoming Leader. State: Replica: 7501d7d926c949ccb4afd126d0dba3d9, State: Running, Role: LEADER
I20260806 00:54:06.268926 1746 consensus_queue.cc:237] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a75dac4a874fecbd60d77ff66f8000" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42859 } } peers { permanent_uuid: "de2288b4d25c41e58dfbc3069a566e35" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42521 } } peers { permanent_uuid: "7501d7d926c949ccb4afd126d0dba3d9" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42251 } }
I20260806 00:54:06.272471 1123 catalog_manager.cc:5495] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7501d7d926c949ccb4afd126d0dba3d9 (127.27.77.194). New cstate: current_term: 1 leader_uuid: "7501d7d926c949ccb4afd126d0dba3d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a75dac4a874fecbd60d77ff66f8000" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de2288b4d25c41e58dfbc3069a566e35" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7501d7d926c949ccb4afd126d0dba3d9" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42251 } health_report { overall_health: HEALTHY } } }
W20260806 00:54:06.306176 1543 tablet.cc:2369] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:54:06.314090 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.310313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52966 (local address 127.27.77.194:42251)
0806 00:54:06.310480 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:06.310484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:06.310499 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:54:06.310582 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:06.310584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:06.310642 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:06.310745 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:54:06.311076 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.311634 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.312474 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:06.312758 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:06.312839 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:06.313178 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:54:06.313616 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:54:06.313718 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:06.313863 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 00:54:06.313921 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 00:54:06.318147 1371 tablet.cc:2369] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:54:06.319170 1717 tablet.cc:2369] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:54:06.320853 1656 raft_consensus.cc:1261] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35 [term 1 FOLLOWER]: Refusing update from remote peer 7501d7d926c949ccb4afd126d0dba3d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:06.320853 1311 raft_consensus.cc:1261] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000 [term 1 FOLLOWER]: Refusing update from remote peer 7501d7d926c949ccb4afd126d0dba3d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:06.321614 1746 consensus_queue.cc:1035] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de2288b4d25c41e58dfbc3069a566e35" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:06.321842 1748 consensus_queue.cc:1035] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a75dac4a874fecbd60d77ff66f8000" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:06.330996 1754 mvcc.cc:205] Tried to move back new op lower bound from 7315364439322005504 to 7315364439120629760. Current Snapshot: MvccSnapshot[applied={T|T < 7315364439322005504}]
I20260806 00:54:06.339962 1124 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:06.340209 1124 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:06.342754 1124 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a95ed172966b4a0cbf4ed26d76e47422: Sending DeleteTablet for 3 replicas of tablet 9f3723caac5248249ea0305fb63b90f7
I20260806 00:54:06.343683 1454 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f3723caac5248249ea0305fb63b90f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:06 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:52954
I20260806 00:54:06.343685 1282 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f3723caac5248249ea0305fb63b90f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:06 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:59204
I20260806 00:54:06.343768 1627 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f3723caac5248249ea0305fb63b90f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:06 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:45758
I20260806 00:54:06.344422 1769 tablet_replica.cc:330] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35: stopping tablet replica
I20260806 00:54:06.344465 1765 tablet_replica.cc:330] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000: stopping tablet replica
I20260806 00:54:06.344797 1769 raft_consensus.cc:2229] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:06.344822 1765 raft_consensus.cc:2229] T 9f3723caac5248249ea0305fb63b90f7 P 21a75dac4a874fecbd60d77ff66f8000 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:06.345081 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 1206
I20260806 00:54:06.345182 1769 raft_consensus.cc:2258] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:06.348023 1766 tablet_replica.cc:330] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9: stopping tablet replica
I20260806 00:54:06.348466 1766 raft_consensus.cc:2229] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:54:06.348932 1766 raft_consensus.cc:2258] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:06.350521 1769 ts_tablet_manager.cc:1905] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:06.351771 1766 ts_tablet_manager.cc:1905] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:06.353856 1769 ts_tablet_manager.cc:1918] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:06.353986 1769 log.cc:1196] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/9f3723caac5248249ea0305fb63b90f7
I20260806 00:54:06.354375 1769 ts_tablet_manager.cc:1939] T 9f3723caac5248249ea0305fb63b90f7 P de2288b4d25c41e58dfbc3069a566e35: Deleting consensus metadata
I20260806 00:54:06.355194 1766 ts_tablet_manager.cc:1918] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:06.355372 1766 log.cc:1196] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/9f3723caac5248249ea0305fb63b90f7
I20260806 00:54:06.355826 1766 ts_tablet_manager.cc:1939] T 9f3723caac5248249ea0305fb63b90f7 P 7501d7d926c949ccb4afd126d0dba3d9: Deleting consensus metadata
I20260806 00:54:06.355906 1105 catalog_manager.cc:4841] TS de2288b4d25c41e58dfbc3069a566e35 (127.27.77.195:42521): tablet 9f3723caac5248249ea0305fb63b90f7 (table test-table [id=64e7e60bdf9d4b65b5698fa1299ddc99]) successfully deleted
I20260806 00:54:06.357133 1104 catalog_manager.cc:4841] TS 7501d7d926c949ccb4afd126d0dba3d9 (127.27.77.194:42251): tablet 9f3723caac5248249ea0305fb63b90f7 (table test-table [id=64e7e60bdf9d4b65b5698fa1299ddc99]) successfully deleted
W20260806 00:54:06.360129 1104 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:42859 (error 108)
I20260806 00:54:06.360363 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 1375
I20260806 00:54:06.365206 1733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.364762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:42859 (local address 127.0.0.1:59206)
0806 00:54:06.364998 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:06.365087 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:42859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260806 00:54:06.365401 1104 catalog_manager.cc:4584] TS 21a75dac4a874fecbd60d77ff66f8000 (127.27.77.193:42859): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9f3723caac5248249ea0305fb63b90f7: Network error: Client connection negotiation failed: client connection to 127.27.77.193:42859: connect: Connection refused (error 111)
I20260806 00:54:06.373368 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 1547
I20260806 00:54:06.385509 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 1084
2026-08-06T00:54:06Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2745 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T00:54:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:06Z Disabled control of system clock
I20260806 00:54:06.422042 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38379
--webserver_interface=127.27.77.254
--webserver_port=0
--builtin_ntp_servers=127.27.77.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:38379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 00:54:06.536783 1775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:06.537148 1775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:06.537194 1775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:06.541365 1775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:06.541429 1775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:06.541445 1775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:06.541462 1775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:06.541477 1775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:54:06.546205 1775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:38379
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:06.547339 1775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:06.548774 1775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:06.557847 1784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:06.557909 1781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:06.557833 1782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:06.558099 1775 server_base.cc:1034] running on GCE node
I20260806 00:54:06.558766 1775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:06.560015 1775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:06.561188 1775 hybrid_clock.cc:648] HybridClock initialized: now 1785977646561158 us; error 50 us; skew 500 ppm
I20260806 00:54:06.563115 1775 webserver.cc:460] Webserver started at http://127.27.77.254:35929/ using document root <none> and password file <none>
I20260806 00:54:06.563808 1775 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:06.563875 1775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:06.564122 1775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:06.566274 1775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "9fd43b26df4c497482effb6652627cee"
format_stamp: "Formatted at 2026-08-06 00:54:06 on dist-test-slave-sb2z"
server_key: "b54f590971883170c6e932a381cbb06c"
server_key_iv: "572d310ccf8bd846e6f7c3ef57242721"
server_key_version: "encryptionkey@0"
I20260806 00:54:06.566876 1775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "9fd43b26df4c497482effb6652627cee"
format_stamp: "Formatted at 2026-08-06 00:54:06 on dist-test-slave-sb2z"
server_key: "b54f590971883170c6e932a381cbb06c"
server_key_iv: "572d310ccf8bd846e6f7c3ef57242721"
server_key_version: "encryptionkey@0"
I20260806 00:54:06.570621 1775 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:54:06.573133 1792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:06.574307 1775 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:54:06.574458 1775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "9fd43b26df4c497482effb6652627cee"
format_stamp: "Formatted at 2026-08-06 00:54:06 on dist-test-slave-sb2z"
server_key: "b54f590971883170c6e932a381cbb06c"
server_key_iv: "572d310ccf8bd846e6f7c3ef57242721"
server_key_version: "encryptionkey@0"
I20260806 00:54:06.574571 1775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:06.579300 1775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:06.579991 1775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:06.580186 1775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:06.587222 1775 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:38379
I20260806 00:54:06.587303 1858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:38379 every 8 connection(s)
I20260806 00:54:06.588295 1775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:06.591832 1860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:06.597659 1860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee: Bootstrap starting.
I20260806 00:54:06.598218 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 1775
I20260806 00:54:06.598361 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:54:06.598639 27959 external_mini_cluster.cc:1428] Setting key 9f6573235ba21b5aecc31889abe19a46
I20260806 00:54:06.600641 1860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:06.601485 1860 log.cc:824] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:06.603722 1860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee: No bootstrap required, opened a new log
I20260806 00:54:06.604399 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.600050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41612 (local address 127.27.77.254:38379)
0806 00:54:06.600718 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:06.600734 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:06.600800 (+ 66us) server_negotiation.cc:407] Connection header received
0806 00:54:06.601468 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:06.601484 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:06.601883 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:06.602212 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:06.603083 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:06.603088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:06.603100 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:06.603111 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:06.603167 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:06.603183 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:06.603303 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:06.603405 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 00:54:06.603679 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":149,"threads_started":1}
I20260806 00:54:06.608217 1860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd43b26df4c497482effb6652627cee" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38379 } }
I20260806 00:54:06.608475 1860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:06.608567 1860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fd43b26df4c497482effb6652627cee, State: Initialized, Role: FOLLOWER
I20260806 00:54:06.609102 1860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd43b26df4c497482effb6652627cee" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38379 } }
I20260806 00:54:06.609329 1860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:06.609417 1860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:06.609558 1860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:06.610553 1860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd43b26df4c497482effb6652627cee" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38379 } }
I20260806 00:54:06.610917 1860 leader_election.cc:302] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fd43b26df4c497482effb6652627cee; no voters:
I20260806 00:54:06.611274 1860 leader_election.cc:288] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:54:06.611380 1866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:06.611661 1866 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [term 1 LEADER]: Becoming Leader. State: Replica: 9fd43b26df4c497482effb6652627cee, State: Running, Role: LEADER
I20260806 00:54:06.611980 1866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd43b26df4c497482effb6652627cee" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38379 } }
I20260806 00:54:06.612757 1860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:06.613845 1868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fd43b26df4c497482effb6652627cee. Latest consensus state: current_term: 1 leader_uuid: "9fd43b26df4c497482effb6652627cee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd43b26df4c497482effb6652627cee" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38379 } } }
I20260806 00:54:06.613999 1868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:06.614293 1867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fd43b26df4c497482effb6652627cee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd43b26df4c497482effb6652627cee" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38379 } } }
I20260806 00:54:06.614393 1867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:06.614610 1876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:06.617565 1876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:06.622874 1876 catalog_manager.cc:1347] Generated new cluster ID: 2d02472bdb1b417bbe1c5a42c2a2fa19
I20260806 00:54:06.622957 1876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:06.643010 1876 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:54:06.643378 1876 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:06.648862 1876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee: Generated new TSK 0
I20260806 00:54:06.649549 1876 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:54:06.652076 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38379
--builtin_ntp_servers=127.27.77.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 00:54:06.769942 1888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:06.770213 1888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:06.770254 1888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:06.774259 1888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:06.774319 1888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:06.774374 1888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:54:06.779201 1888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:06.780359 1888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:06.781785 1888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:06.791467 1893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:06.791553 1894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:06.791558 1896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:06.791606 1888 server_base.cc:1034] running on GCE node
I20260806 00:54:06.792156 1888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:06.792812 1888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:06.794003 1888 hybrid_clock.cc:648] HybridClock initialized: now 1785977646793971 us; error 46 us; skew 500 ppm
I20260806 00:54:06.795934 1888 webserver.cc:460] Webserver started at http://127.27.77.193:42091/ using document root <none> and password file <none>
I20260806 00:54:06.796527 1888 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:06.796628 1888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:06.796865 1888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:06.798950 1888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "d6f489bee9ca49cbb135aac4d7ca4559"
format_stamp: "Formatted at 2026-08-06 00:54:06 on dist-test-slave-sb2z"
server_key: "474962ec044f154b47a5f581813bfda0"
server_key_iv: "b459810cf068ccc7b0fcbb5c1b316bbf"
server_key_version: "encryptionkey@0"
I20260806 00:54:06.799579 1888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "d6f489bee9ca49cbb135aac4d7ca4559"
format_stamp: "Formatted at 2026-08-06 00:54:06 on dist-test-slave-sb2z"
server_key: "474962ec044f154b47a5f581813bfda0"
server_key_iv: "b459810cf068ccc7b0fcbb5c1b316bbf"
server_key_version: "encryptionkey@0"
I20260806 00:54:06.803208 1888 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:54:06.805747 1903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:06.806960 1888 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:54:06.807114 1888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "d6f489bee9ca49cbb135aac4d7ca4559"
format_stamp: "Formatted at 2026-08-06 00:54:06 on dist-test-slave-sb2z"
server_key: "474962ec044f154b47a5f581813bfda0"
server_key_iv: "b459810cf068ccc7b0fcbb5c1b316bbf"
server_key_version: "encryptionkey@0"
I20260806 00:54:06.807233 1888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:06.811913 1888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:06.812551 1888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:06.812747 1888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:06.813422 1888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:06.814415 1888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:06.814487 1888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:06.814550 1888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:06.814580 1888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:06.824658 1888 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:36823
I20260806 00:54:06.824697 2051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:36823 every 8 connection(s)
I20260806 00:54:06.825634 1888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:54:06.829366 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 1888
I20260806 00:54:06.829483 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:54:06.829843 27959 external_mini_cluster.cc:1428] Setting key 6d6348c62e653f616d8fdfabab11d78a
I20260806 00:54:06.832346 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38379
--builtin_ntp_servers=127.27.77.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 00:54:06.832803 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.827679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:38121 (local address 127.27.77.254:38379)
0806 00:54:06.827835 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:06.827838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:06.828950 (+ 1112us) server_negotiation.cc:407] Connection header received
0806 00:54:06.830625 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:06.830629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:06.830679 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:06.830785 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:06.832212 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:06.832220 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:06.832227 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:06.832243 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:06.832272 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:06.832284 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:06.832395 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:06.832623 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 00:54:06.832672 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:54:06.833688 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:06.828040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38379 (local address 127.27.77.193:38121)
0806 00:54:06.828787 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:06.828830 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:06.830395 (+ 1565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:06.830985 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:06.830994 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:06.831105 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:06.831873 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:54:06.831875 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:06.831928 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:54:06.832016 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:06.832393 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:06.832402 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:06.832407 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 00:54:06.832657 (+ 250us) client_negotiation.cc:240] Negotiation successful
0806 00:54:06.832907 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":143,"threads_started":1}
I20260806 00:54:06.835083 2052 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38379
I20260806 00:54:06.835338 2052 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:06.835882 2052 heartbeater.cc:511] Master 127.27.77.254:38379 requested a full tablet report, sending...
I20260806 00:54:06.837623 1812 ts_manager.cc:194] Registered new tserver with Master: d6f489bee9ca49cbb135aac4d7ca4559 (127.27.77.193:36823)
W20260806 00:54:06.947398 2055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:06.947669 2055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:06.947711 2055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:06.952035 2055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:06.952106 2055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:06.952164 2055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:54:06.957629 2055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:06.958735 2055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:06.960276 2055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:06.970971 2065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:06.971022 2060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:06.971005 2062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:06.971551 2055 server_base.cc:1034] running on GCE node
I20260806 00:54:06.972075 2055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:06.972878 2055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:06.974092 2055 hybrid_clock.cc:648] HybridClock initialized: now 1785977646974025 us; error 78 us; skew 500 ppm
I20260806 00:54:06.976649 2055 webserver.cc:460] Webserver started at http://127.27.77.194:38595/ using document root <none> and password file <none>
I20260806 00:54:06.977412 2055 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:06.977515 2055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:06.977769 2055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:06.980698 2055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "f5d2ce607b604b7192bb429d5ebd8731"
format_stamp: "Formatted at 2026-08-06 00:54:06 on dist-test-slave-sb2z"
server_key: "2b1f516841bda28517bcfdf19432a05a"
server_key_iv: "ed350cabf221b966e0149f5551768707"
server_key_version: "encryptionkey@0"
I20260806 00:54:06.981370 2055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "f5d2ce607b604b7192bb429d5ebd8731"
format_stamp: "Formatted at 2026-08-06 00:54:06 on dist-test-slave-sb2z"
server_key: "2b1f516841bda28517bcfdf19432a05a"
server_key_iv: "ed350cabf221b966e0149f5551768707"
server_key_version: "encryptionkey@0"
I20260806 00:54:06.985529 2055 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:54:06.989560 2074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:06.991176 2055 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:54:06.991329 2055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "f5d2ce607b604b7192bb429d5ebd8731"
format_stamp: "Formatted at 2026-08-06 00:54:06 on dist-test-slave-sb2z"
server_key: "2b1f516841bda28517bcfdf19432a05a"
server_key_iv: "ed350cabf221b966e0149f5551768707"
server_key_version: "encryptionkey@0"
I20260806 00:54:06.991477 2055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:06.996394 2055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:06.997043 2055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:06.997251 2055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:06.997840 2055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:06.998781 2055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:06.998832 2055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:06.998870 2055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:06.998885 2055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:07.008589 2055 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:43497
I20260806 00:54:07.008617 2220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:43497 every 8 connection(s)
I20260806 00:54:07.009598 2055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:54:07.010355 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 2055
I20260806 00:54:07.010496 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:54:07.010808 27959 external_mini_cluster.cc:1428] Setting key 01357b426b9788af3d96d7dbbe188a70
I20260806 00:54:07.013309 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38379
--builtin_ntp_servers=127.27.77.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 00:54:07.017055 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.011839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:37705 (local address 127.27.77.254:38379)
0806 00:54:07.012001 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:07.012006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:07.013363 (+ 1357us) server_negotiation.cc:407] Connection header received
0806 00:54:07.015016 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:07.015020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:07.015070 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:07.015151 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:07.016399 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:07.016403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:07.016406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:07.016418 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:07.016440 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:07.016456 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:07.016608 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:07.016899 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 00:54:07.016944 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 00:54:07.018198 2223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.012441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38379 (local address 127.27.77.194:37705)
0806 00:54:07.013218 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:07.013259 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:07.014783 (+ 1524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:07.015325 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:07.015332 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:07.015420 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:07.016124 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:54:07.016126 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:07.016168 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:54:07.016238 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:07.016676 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:07.016699 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:07.016712 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 00:54:07.016913 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 00:54:07.017163 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":176,"threads_started":1}
I20260806 00:54:07.019505 2221 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38379
I20260806 00:54:07.019802 2221 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:07.020327 2221 heartbeater.cc:511] Master 127.27.77.254:38379 requested a full tablet report, sending...
I20260806 00:54:07.021611 1812 ts_manager.cc:194] Registered new tserver with Master: f5d2ce607b604b7192bb429d5ebd8731 (127.27.77.194:43497)
W20260806 00:54:07.130621 2224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:07.130872 2224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:07.130949 2224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:07.134974 2224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:07.135035 2224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:07.135092 2224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:54:07.140663 2224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:07.141819 2224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:07.143268 2224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:07.153035 2230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:07.153071 2229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:07.153065 2233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:07.153283 2224 server_base.cc:1034] running on GCE node
I20260806 00:54:07.153743 2224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:07.154368 2224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:07.155673 2224 hybrid_clock.cc:648] HybridClock initialized: now 1785977647155656 us; error 24 us; skew 500 ppm
I20260806 00:54:07.157546 2224 webserver.cc:460] Webserver started at http://127.27.77.195:34859/ using document root <none> and password file <none>
I20260806 00:54:07.158133 2224 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:07.158197 2224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:07.158416 2224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:07.160604 2224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "7e1b4c902d044ce4bf4d41dc07153204"
format_stamp: "Formatted at 2026-08-06 00:54:07 on dist-test-slave-sb2z"
server_key: "45cb3a6a4997356907d9c944e96f7214"
server_key_iv: "5ac68bb7ad4eed127b0ac4282fb4e4a7"
server_key_version: "encryptionkey@0"
I20260806 00:54:07.161193 2224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "7e1b4c902d044ce4bf4d41dc07153204"
format_stamp: "Formatted at 2026-08-06 00:54:07 on dist-test-slave-sb2z"
server_key: "45cb3a6a4997356907d9c944e96f7214"
server_key_iv: "5ac68bb7ad4eed127b0ac4282fb4e4a7"
server_key_version: "encryptionkey@0"
I20260806 00:54:07.165027 2224 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:54:07.167738 2239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:07.168988 2224 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:54:07.169142 2224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "7e1b4c902d044ce4bf4d41dc07153204"
format_stamp: "Formatted at 2026-08-06 00:54:07 on dist-test-slave-sb2z"
server_key: "45cb3a6a4997356907d9c944e96f7214"
server_key_iv: "5ac68bb7ad4eed127b0ac4282fb4e4a7"
server_key_version: "encryptionkey@0"
I20260806 00:54:07.169262 2224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:07.173957 2224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:07.174602 2224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:07.174795 2224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:07.175362 2224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:07.176359 2224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:07.176433 2224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:07.176502 2224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:07.176555 2224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:07.186280 2224 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:43695
I20260806 00:54:07.186307 2395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:43695 every 8 connection(s)
I20260806 00:54:07.187243 2224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:54:07.191671 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 2224
I20260806 00:54:07.191815 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:54:07.192097 27959 external_mini_cluster.cc:1428] Setting key 6fe1104063bd1f432df3e36ec345583e
I20260806 00:54:07.193796 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.189163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:33149 (local address 127.27.77.254:38379)
0806 00:54:07.189304 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:07.189308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:07.190384 (+ 1076us) server_negotiation.cc:407] Connection header received
0806 00:54:07.191854 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:07.191857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:07.191907 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:07.192023 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:07.193240 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:07.193244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:07.193246 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:07.193259 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:07.193286 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:07.193295 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:07.193388 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:07.193582 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 00:54:07.193666 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:54:07.194914 2399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.189554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38379 (local address 127.27.77.195:33149)
0806 00:54:07.190230 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:07.190270 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:07.191634 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:07.192215 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:07.192224 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:07.192326 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:07.192930 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:54:07.192931 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:07.192992 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:54:07.193083 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:07.193379 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:07.193388 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:07.193396 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 00:54:07.193603 (+ 207us) client_negotiation.cc:240] Negotiation successful
0806 00:54:07.193827 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":125,"threads_started":1}
I20260806 00:54:07.196132 2396 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38379
I20260806 00:54:07.196378 2396 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:07.196810 2396 heartbeater.cc:511] Master 127.27.77.254:38379 requested a full tablet report, sending...
I20260806 00:54:07.197817 1812 ts_manager.cc:194] Registered new tserver with Master: 7e1b4c902d044ce4bf4d41dc07153204 (127.27.77.195:43695)
I20260806 00:54:07.206653 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:54:07.210444 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.208780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41624 (local address 127.27.77.254:38379)
0806 00:54:07.208951 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:07.208957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:07.209114 (+ 157us) server_negotiation.cc:407] Connection header received
0806 00:54:07.209188 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:07.209190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:07.209228 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:07.209306 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:07.209959 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:07.209966 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:07.209969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:07.209982 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:07.210012 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:07.210020 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:07.210094 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:07.210237 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 00:54:07.210273 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 00:54:07.213546 1812 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:54:07.216749 1812 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:54:07.228962 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.226109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:43695 (local address 127.0.0.1:34800)
0806 00:54:07.226447 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:07.226536 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:07.226615 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:07.227258 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:07.227263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:07.227342 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:07.228031 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:54:07.228033 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:07.228057 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:54:07.228138 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:07.228569 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:07.228581 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:07.228589 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 00:54:07.228718 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 00:54:07.228791 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":90,"threads_started":1}
I20260806 00:54:07.229019 2411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.225744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:36823 (local address 127.0.0.1:36748)
0806 00:54:07.226438 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:07.226495 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:07.226615 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:07.227441 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:07.227444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:07.227459 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:07.228151 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:54:07.228153 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:07.228420 (+ 267us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:54:07.228450 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:07.228832 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:07.228835 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:07.228838 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 00:54:07.228890 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 00:54:07.228916 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":31,"thread_start_us":102,"threads_started":1}
I20260806 00:54:07.229327 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.225554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46252 (local address 127.27.77.194:43497)
0806 00:54:07.226120 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:07.226136 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:07.226828 (+ 692us) server_negotiation.cc:407] Connection header received
0806 00:54:07.226894 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:07.226901 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:07.227107 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:07.227241 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:07.228606 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:07.228617 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:07.228625 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:07.228637 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:07.228693 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:07.228735 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:07.228843 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:07.229096 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 00:54:07.229169 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":172,"threads_started":1}
I20260806 00:54:07.229689 2409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.225344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:43497 (local address 127.0.0.1:46252)
0806 00:54:07.226642 (+ 1298us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:07.226658 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:07.226730 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:07.227977 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:07.227980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:07.227994 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:07.228426 (+ 432us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:54:07.228428 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:07.228436 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:54:07.228452 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:07.228890 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:07.228893 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:07.228896 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 00:54:07.228971 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 00:54:07.229008 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":115,"threads_started":1}
I20260806 00:54:07.229790 2414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.226307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34800 (local address 127.27.77.195:43695)
0806 00:54:07.226883 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:07.226894 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:07.226913 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:54:07.226959 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:07.226963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:07.227094 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:07.227240 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:07.228280 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:07.228289 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:07.228306 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:07.228323 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:07.228374 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:07.228408 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:07.228519 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:07.228765 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 00:54:07.228851 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":155,"threads_started":1}
I20260806 00:54:07.229926 2412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.225982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36748 (local address 127.27.77.193:36823)
0806 00:54:07.227032 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:07.227041 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:07.227062 (+ 21us) server_negotiation.cc:407] Connection header received
0806 00:54:07.227110 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:07.227114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:07.227294 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:07.227998 (+ 704us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:07.228624 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:07.228629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:07.228639 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:07.228654 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:07.228698 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:07.228727 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:07.229638 (+ 911us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:07.229691 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 00:54:07.229758 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":950,"thread_start_us":96,"threads_started":1}
I20260806 00:54:07.232667 2140 tablet_service.cc:1467] Processing CreateTablet for tablet 8c3c977f65e046589e1e35661b7c5631 (DEFAULT_TABLE table=test-table [id=18b05aac03b64615821a082248cc50af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:07.232663 2304 tablet_service.cc:1467] Processing CreateTablet for tablet 8c3c977f65e046589e1e35661b7c5631 (DEFAULT_TABLE table=test-table [id=18b05aac03b64615821a082248cc50af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:07.232666 1970 tablet_service.cc:1467] Processing CreateTablet for tablet 8c3c977f65e046589e1e35661b7c5631 (DEFAULT_TABLE table=test-table [id=18b05aac03b64615821a082248cc50af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:07.233788 2140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c3c977f65e046589e1e35661b7c5631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:07.233812 2304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c3c977f65e046589e1e35661b7c5631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:07.233812 1970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c3c977f65e046589e1e35661b7c5631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:07.240499 2416 tablet_bootstrap.cc:492] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731: Bootstrap starting.
I20260806 00:54:07.241677 2418 tablet_bootstrap.cc:492] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204: Bootstrap starting.
I20260806 00:54:07.241677 2417 tablet_bootstrap.cc:492] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559: Bootstrap starting.
I20260806 00:54:07.243788 2416 tablet_bootstrap.cc:654] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:07.243947 2417 tablet_bootstrap.cc:654] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:07.243947 2418 tablet_bootstrap.cc:654] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:07.244644 2416 log.cc:824] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:07.244832 2417 log.cc:824] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:07.244832 2418 log.cc:824] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:07.246696 2418 tablet_bootstrap.cc:492] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204: No bootstrap required, opened a new log
I20260806 00:54:07.246822 2417 tablet_bootstrap.cc:492] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559: No bootstrap required, opened a new log
I20260806 00:54:07.246822 2416 tablet_bootstrap.cc:492] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731: No bootstrap required, opened a new log
I20260806 00:54:07.246973 2418 ts_tablet_manager.cc:1397] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:54:07.247033 2417 ts_tablet_manager.cc:1397] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:54:07.247021 2416 ts_tablet_manager.cc:1397] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20260806 00:54:07.250626 2418 raft_consensus.cc:348] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5d2ce607b604b7192bb429d5ebd8731" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43497 } } peers { permanent_uuid: "7e1b4c902d044ce4bf4d41dc07153204" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43695 } } peers { permanent_uuid: "d6f489bee9ca49cbb135aac4d7ca4559" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36823 } }
I20260806 00:54:07.250897 2418 raft_consensus.cc:374] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:07.250954 2418 raft_consensus.cc:729] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e1b4c902d044ce4bf4d41dc07153204, State: Initialized, Role: FOLLOWER
I20260806 00:54:07.251431 2418 consensus_queue.cc:260] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5d2ce607b604b7192bb429d5ebd8731" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43497 } } peers { permanent_uuid: "7e1b4c902d044ce4bf4d41dc07153204" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43695 } } peers { permanent_uuid: "d6f489bee9ca49cbb135aac4d7ca4559" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36823 } }
I20260806 00:54:07.251719 2416 raft_consensus.cc:348] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5d2ce607b604b7192bb429d5ebd8731" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43497 } } peers { permanent_uuid: "7e1b4c902d044ce4bf4d41dc07153204" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43695 } } peers { permanent_uuid: "d6f489bee9ca49cbb135aac4d7ca4559" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36823 } }
I20260806 00:54:07.251719 2417 raft_consensus.cc:348] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5d2ce607b604b7192bb429d5ebd8731" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43497 } } peers { permanent_uuid: "7e1b4c902d044ce4bf4d41dc07153204" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43695 } } peers { permanent_uuid: "d6f489bee9ca49cbb135aac4d7ca4559" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36823 } }
I20260806 00:54:07.251968 2417 raft_consensus.cc:374] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:07.251968 2416 raft_consensus.cc:374] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:07.252043 2416 raft_consensus.cc:729] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5d2ce607b604b7192bb429d5ebd8731, State: Initialized, Role: FOLLOWER
I20260806 00:54:07.252053 2417 raft_consensus.cc:729] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6f489bee9ca49cbb135aac4d7ca4559, State: Initialized, Role: FOLLOWER
I20260806 00:54:07.252307 2396 heartbeater.cc:503] Master 127.27.77.254:38379 was elected leader, sending a full tablet report...
I20260806 00:54:07.252513 2417 consensus_queue.cc:260] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5d2ce607b604b7192bb429d5ebd8731" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43497 } } peers { permanent_uuid: "7e1b4c902d044ce4bf4d41dc07153204" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43695 } } peers { permanent_uuid: "d6f489bee9ca49cbb135aac4d7ca4559" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36823 } }
I20260806 00:54:07.252523 2416 consensus_queue.cc:260] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5d2ce607b604b7192bb429d5ebd8731" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43497 } } peers { permanent_uuid: "7e1b4c902d044ce4bf4d41dc07153204" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43695 } } peers { permanent_uuid: "d6f489bee9ca49cbb135aac4d7ca4559" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36823 } }
I20260806 00:54:07.252645 2418 ts_tablet_manager.cc:1428] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260806 00:54:07.253180 2052 heartbeater.cc:503] Master 127.27.77.254:38379 was elected leader, sending a full tablet report...
I20260806 00:54:07.253676 2417 ts_tablet_manager.cc:1428] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 00:54:07.253839 2221 heartbeater.cc:503] Master 127.27.77.254:38379 was elected leader, sending a full tablet report...
I20260806 00:54:07.254406 2416 ts_tablet_manager.cc:1428] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
W20260806 00:54:07.260632 2222 tablet.cc:2369] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:54:07.326857 2053 tablet.cc:2369] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:54:07.438410 2397 tablet.cc:2369] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:54:07.534585 2425 raft_consensus.cc:482] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:54:07.534825 2425 raft_consensus.cc:504] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5d2ce607b604b7192bb429d5ebd8731" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43497 } } peers { permanent_uuid: "7e1b4c902d044ce4bf4d41dc07153204" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43695 } } peers { permanent_uuid: "d6f489bee9ca49cbb135aac4d7ca4559" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36823 } }
I20260806 00:54:07.536253 2425 leader_election.cc:288] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5d2ce607b604b7192bb429d5ebd8731 (127.27.77.194:43497), 7e1b4c902d044ce4bf4d41dc07153204 (127.27.77.195:43695)
I20260806 00:54:07.538762 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.536712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:36349 (local address 127.27.77.194:43497)
0806 00:54:07.536869 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:07.536873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:07.536938 (+ 65us) server_negotiation.cc:407] Connection header received
0806 00:54:07.537092 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:07.537096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:07.537159 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:07.537279 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:07.538261 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:07.538265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:07.538268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:07.538281 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:07.538305 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:07.538318 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:07.538412 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:07.538532 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 00:54:07.538589 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:54:07.538748 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.536530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:43497 (local address 127.27.77.193:36349)
0806 00:54:07.536838 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:07.536856 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:07.536969 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:07.537345 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:07.537348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:07.537369 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:07.538049 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:54:07.538053 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:07.538068 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:54:07.538105 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:07.538421 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:07.538424 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:07.538428 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 00:54:07.538523 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 00:54:07.538567 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":125,"threads_started":1}
I20260806 00:54:07.538885 2414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.537200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:38421 (local address 127.27.77.195:43695)
0806 00:54:07.537354 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:07.537359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:07.537374 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:54:07.537519 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:07.537524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:07.537574 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:07.537694 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:07.538429 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:07.538432 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:07.538435 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:07.538451 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:07.538475 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:07.538519 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:07.538608 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:07.538695 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 00:54:07.538735 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:54:07.539306 2428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.536978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:43695 (local address 127.27.77.193:38421)
0806 00:54:07.537301 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:07.537317 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:07.537399 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:07.537708 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:07.537711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:07.537726 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:07.538231 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:54:07.538233 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:07.538245 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:54:07.538272 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:07.538608 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:07.538612 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:07.538615 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 00:54:07.538701 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 00:54:07.538735 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":93,"threads_started":1}
I20260806 00:54:07.539613 2165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c3c977f65e046589e1e35661b7c5631" candidate_uuid: "d6f489bee9ca49cbb135aac4d7ca4559" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5d2ce607b604b7192bb429d5ebd8731" is_pre_election: true
I20260806 00:54:07.539961 2165 raft_consensus.cc:2454] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6f489bee9ca49cbb135aac4d7ca4559 in term 0.
I20260806 00:54:07.539990 2328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c3c977f65e046589e1e35661b7c5631" candidate_uuid: "d6f489bee9ca49cbb135aac4d7ca4559" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e1b4c902d044ce4bf4d41dc07153204" is_pre_election: true
I20260806 00:54:07.540273 2328 raft_consensus.cc:2454] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6f489bee9ca49cbb135aac4d7ca4559 in term 0.
I20260806 00:54:07.540566 1905 leader_election.cc:302] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6f489bee9ca49cbb135aac4d7ca4559, f5d2ce607b604b7192bb429d5ebd8731; no voters:
I20260806 00:54:07.540858 2425 raft_consensus.cc:2781] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:54:07.540987 2425 raft_consensus.cc:482] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:54:07.541054 2425 raft_consensus.cc:3037] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:07.542173 2425 raft_consensus.cc:504] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5d2ce607b604b7192bb429d5ebd8731" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43497 } } peers { permanent_uuid: "7e1b4c902d044ce4bf4d41dc07153204" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43695 } } peers { permanent_uuid: "d6f489bee9ca49cbb135aac4d7ca4559" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36823 } }
I20260806 00:54:07.542526 2425 leader_election.cc:288] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [CANDIDATE]: Term 1 election: Requested vote from peers f5d2ce607b604b7192bb429d5ebd8731 (127.27.77.194:43497), 7e1b4c902d044ce4bf4d41dc07153204 (127.27.77.195:43695)
I20260806 00:54:07.542904 2165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c3c977f65e046589e1e35661b7c5631" candidate_uuid: "d6f489bee9ca49cbb135aac4d7ca4559" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5d2ce607b604b7192bb429d5ebd8731"
I20260806 00:54:07.542938 2328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c3c977f65e046589e1e35661b7c5631" candidate_uuid: "d6f489bee9ca49cbb135aac4d7ca4559" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e1b4c902d044ce4bf4d41dc07153204"
I20260806 00:54:07.543051 2165 raft_consensus.cc:3037] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:07.543066 2328 raft_consensus.cc:3037] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:07.544216 2328 raft_consensus.cc:2454] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6f489bee9ca49cbb135aac4d7ca4559 in term 1.
I20260806 00:54:07.544304 2165 raft_consensus.cc:2454] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6f489bee9ca49cbb135aac4d7ca4559 in term 1.
I20260806 00:54:07.544518 1905 leader_election.cc:302] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e1b4c902d044ce4bf4d41dc07153204, d6f489bee9ca49cbb135aac4d7ca4559; no voters:
I20260806 00:54:07.544715 2425 raft_consensus.cc:2781] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:07.544950 2425 raft_consensus.cc:686] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [term 1 LEADER]: Becoming Leader. State: Replica: d6f489bee9ca49cbb135aac4d7ca4559, State: Running, Role: LEADER
I20260806 00:54:07.545295 2425 consensus_queue.cc:237] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5d2ce607b604b7192bb429d5ebd8731" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43497 } } peers { permanent_uuid: "7e1b4c902d044ce4bf4d41dc07153204" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43695 } } peers { permanent_uuid: "d6f489bee9ca49cbb135aac4d7ca4559" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36823 } }
I20260806 00:54:07.548746 1812 catalog_manager.cc:5495] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6f489bee9ca49cbb135aac4d7ca4559 (127.27.77.193). New cstate: current_term: 1 leader_uuid: "d6f489bee9ca49cbb135aac4d7ca4559" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5d2ce607b604b7192bb429d5ebd8731" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e1b4c902d044ce4bf4d41dc07153204" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6f489bee9ca49cbb135aac4d7ca4559" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36823 } health_report { overall_health: HEALTHY } } }
I20260806 00:54:07.595184 2412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.593420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36752 (local address 127.27.77.193:36823)
0806 00:54:07.593577 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:07.593582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:07.593627 (+ 45us) server_negotiation.cc:407] Connection header received
0806 00:54:07.593715 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:07.593718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:07.593777 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:07.593868 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:07.594704 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:07.594707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:07.594709 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:07.594721 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:07.594740 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:07.594748 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:07.594847 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:07.595003 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 00:54:07.595048 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 00:54:07.601368 2328 raft_consensus.cc:1261] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204 [term 1 FOLLOWER]: Refusing update from remote peer d6f489bee9ca49cbb135aac4d7ca4559: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:07.601416 2165 raft_consensus.cc:1261] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731 [term 1 FOLLOWER]: Refusing update from remote peer d6f489bee9ca49cbb135aac4d7ca4559: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:07.602169 2425 consensus_queue.cc:1035] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e1b4c902d044ce4bf4d41dc07153204" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:07.602423 2429 consensus_queue.cc:1035] T 8c3c977f65e046589e1e35661b7c5631 P d6f489bee9ca49cbb135aac4d7ca4559 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5d2ce607b604b7192bb429d5ebd8731" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:07.610569 2435 mvcc.cc:205] Tried to move back new op lower bound from 7315364444567158784 to 7315364444348006400. Current Snapshot: MvccSnapshot[applied={T|T < 7315364444567158784}]
I20260806 00:54:07.611382 2434 mvcc.cc:205] Tried to move back new op lower bound from 7315364444567158784 to 7315364444348006400. Current Snapshot: MvccSnapshot[applied={T|T < 7315364444567158784}]
I20260806 00:54:07.620064 1812 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:07.620278 1812 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:07.622712 1812 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fd43b26df4c497482effb6652627cee: Sending DeleteTablet for 3 replicas of tablet 8c3c977f65e046589e1e35661b7c5631
I20260806 00:54:07.623584 1970 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c3c977f65e046589e1e35661b7c5631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:07 UTC) from {username='slave'} at 127.0.0.1:36748
I20260806 00:54:07.623793 2140 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c3c977f65e046589e1e35661b7c5631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:07 UTC) from {username='slave'} at 127.0.0.1:46252
I20260806 00:54:07.623807 2304 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c3c977f65e046589e1e35661b7c5631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:07 UTC) from {username='slave'} at 127.0.0.1:34800
I20260806 00:54:07.624296 2451 tablet_replica.cc:330] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731: stopping tablet replica
I20260806 00:54:07.624594 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 1888
I20260806 00:54:07.628537 2455 tablet_replica.cc:330] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204: stopping tablet replica
I20260806 00:54:07.628999 2455 raft_consensus.cc:2229] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:07.629442 2455 raft_consensus.cc:2258] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:07.624639 2451 raft_consensus.cc:2229] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:07.630139 2451 raft_consensus.cc:2258] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:07.632155 2451 ts_tablet_manager.cc:1905] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:07.635857 2451 ts_tablet_manager.cc:1918] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:07.635980 2451 log.cc:1196] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/8c3c977f65e046589e1e35661b7c5631
I20260806 00:54:07.636333 2451 ts_tablet_manager.cc:1939] T 8c3c977f65e046589e1e35661b7c5631 P f5d2ce607b604b7192bb429d5ebd8731: Deleting consensus metadata
I20260806 00:54:07.636906 2455 ts_tablet_manager.cc:1905] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:07.638154 1794 catalog_manager.cc:4841] TS f5d2ce607b604b7192bb429d5ebd8731 (127.27.77.194:43497): tablet 8c3c977f65e046589e1e35661b7c5631 (table test-table [id=18b05aac03b64615821a082248cc50af]) successfully deleted
I20260806 00:54:07.639318 2455 ts_tablet_manager.cc:1918] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:07.639441 2455 log.cc:1196] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/8c3c977f65e046589e1e35661b7c5631
I20260806 00:54:07.639773 2455 ts_tablet_manager.cc:1939] T 8c3c977f65e046589e1e35661b7c5631 P 7e1b4c902d044ce4bf4d41dc07153204: Deleting consensus metadata
W20260806 00:54:07.640532 1793 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:36823 (error 108)
I20260806 00:54:07.641085 1794 catalog_manager.cc:4841] TS 7e1b4c902d044ce4bf4d41dc07153204 (127.27.77.195:43695): tablet 8c3c977f65e046589e1e35661b7c5631 (table test-table [id=18b05aac03b64615821a082248cc50af]) successfully deleted
I20260806 00:54:07.641506 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 2055
I20260806 00:54:07.641686 2409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.641355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:36823 (local address 127.0.0.1:36760)
0806 00:54:07.641495 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:07.641572 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 00:54:07.647709 1793 catalog_manager.cc:4584] TS d6f489bee9ca49cbb135aac4d7ca4559 (127.27.77.193:36823): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8c3c977f65e046589e1e35661b7c5631: Network error: Client connection negotiation failed: client connection to 127.27.77.193:36823: connect: Connection refused (error 111)
I20260806 00:54:07.651279 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 2224
I20260806 00:54:07.660398 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 1775
2026-08-06T00:54:07Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1275 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 00:54:07.689149 27959 mini_oidc.cc:152] Starting JWKS server
I20260806 00:54:07.690862 27959 webserver.cc:460] Webserver started at https://127.0.0.1:36673/ using document root <none> and password file <none>
I20260806 00:54:07.691002 27959 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 00:54:07.691224 27959 webserver.cc:460] Webserver started at http://127.0.0.1:44559/ using document root <none> and password file <none>
2026-08-06T00:54:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:07Z Disabled control of system clock
I20260806 00:54:07.702322 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:41881
--webserver_interface=127.27.77.254
--webserver_port=0
--builtin_ntp_servers=127.27.77.212:43875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:41881
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36673/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/testchainca.cert with env {}
W20260806 00:54:07.817301 2465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:07.817569 2465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:07.817608 2465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:07.821507 2465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:07.821565 2465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:07.821580 2465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:07.821597 2465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:07.821612 2465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 00:54:07.821628 2465 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 00:54:07.821641 2465 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36673/jwks.json
I20260806 00:54:07.826913 2465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:41881
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:41881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36673/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:07.827965 2465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:07.828926 2465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:54:07.838003 2465 server_base.cc:1034] running on GCE node
W20260806 00:54:07.837944 2475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:07.837913 2474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:07.838321 2478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:07.838971 2465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:07.840197 2465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:07.841410 2465 hybrid_clock.cc:648] HybridClock initialized: now 1785977647841389 us; error 42 us; skew 500 ppm
I20260806 00:54:07.843439 2465 webserver.cc:460] Webserver started at http://127.27.77.254:40311/ using document root <none> and password file <none>
I20260806 00:54:07.844067 2465 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:07.844170 2465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:07.844408 2465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:07.846776 2465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "9bb581623dfb4b8b89f0467f57887e70"
format_stamp: "Formatted at 2026-08-06 00:54:07 on dist-test-slave-sb2z"
server_key: "1ee942a63bbbd792afd6c4a0a8f6032f"
server_key_iv: "c02db23f7eae8049ba301dc0b0b5f00a"
server_key_version: "encryptionkey@0"
I20260806 00:54:07.847389 2465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "9bb581623dfb4b8b89f0467f57887e70"
format_stamp: "Formatted at 2026-08-06 00:54:07 on dist-test-slave-sb2z"
server_key: "1ee942a63bbbd792afd6c4a0a8f6032f"
server_key_iv: "c02db23f7eae8049ba301dc0b0b5f00a"
server_key_version: "encryptionkey@0"
I20260806 00:54:07.851475 2465 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 00:54:07.854216 2485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:07.855800 2465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:54:07.855952 2465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "9bb581623dfb4b8b89f0467f57887e70"
format_stamp: "Formatted at 2026-08-06 00:54:07 on dist-test-slave-sb2z"
server_key: "1ee942a63bbbd792afd6c4a0a8f6032f"
server_key_iv: "c02db23f7eae8049ba301dc0b0b5f00a"
server_key_version: "encryptionkey@0"
I20260806 00:54:07.856074 2465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:07.874938 2465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:07.875751 2465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:07.875969 2465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:07.883394 2465 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:41881
I20260806 00:54:07.883404 2553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:41881 every 8 connection(s)
I20260806 00:54:07.884435 2465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:07.887814 2554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:07.888710 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 2465
I20260806 00:54:07.888863 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:54:07.889153 27959 external_mini_cluster.cc:1428] Setting key 34c3688c1191fdb885fcee8a82dc2905
I20260806 00:54:07.895318 2554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70: Bootstrap starting.
I20260806 00:54:07.897840 2554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:07.898528 2557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.890477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53500 (local address 127.27.77.254:41881)
0806 00:54:07.891138 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:07.891154 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:07.891191 (+ 37us) server_negotiation.cc:407] Connection header received
0806 00:54:07.891977 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:07.891998 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:07.892396 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:07.892753 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:07.893188 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:07.894125 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:07.895075 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:07.895635 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:07.895807 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:07.896735 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:07.896740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:07.896751 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:07.897164 (+ 413us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:07.897222 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:07.897243 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:07.897362 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:07.897521 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 00:54:07.897777 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":114,"threads_started":1}
I20260806 00:54:07.898620 2554 log.cc:824] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:07.900722 2554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70: No bootstrap required, opened a new log
I20260806 00:54:07.901368 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:54:07.904645 2554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb581623dfb4b8b89f0467f57887e70" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41881 } }
I20260806 00:54:07.904884 2554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:07.904974 2554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb581623dfb4b8b89f0467f57887e70, State: Initialized, Role: FOLLOWER
I20260806 00:54:07.905457 2554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb581623dfb4b8b89f0467f57887e70" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41881 } }
I20260806 00:54:07.905637 2554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:07.905704 2554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:07.905803 2554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:07.906679 2554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb581623dfb4b8b89f0467f57887e70" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41881 } }
I20260806 00:54:07.907028 2554 leader_election.cc:302] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bb581623dfb4b8b89f0467f57887e70; no voters:
I20260806 00:54:07.907373 2554 leader_election.cc:288] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:54:07.907480 2561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:07.907807 2561 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [term 1 LEADER]: Becoming Leader. State: Replica: 9bb581623dfb4b8b89f0467f57887e70, State: Running, Role: LEADER
I20260806 00:54:07.908133 2561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb581623dfb4b8b89f0467f57887e70" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41881 } }
I20260806 00:54:07.908694 2554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:07.909881 2562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bb581623dfb4b8b89f0467f57887e70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb581623dfb4b8b89f0467f57887e70" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41881 } } }
I20260806 00:54:07.909881 2563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bb581623dfb4b8b89f0467f57887e70. Latest consensus state: current_term: 1 leader_uuid: "9bb581623dfb4b8b89f0467f57887e70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bb581623dfb4b8b89f0467f57887e70" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41881 } } }
I20260806 00:54:07.910043 2562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:07.910043 2563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:07.910602 2572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:07.913476 2572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:07.918711 2572 catalog_manager.cc:1347] Generated new cluster ID: 1394c1ae0f844043831b9c8f24128e75
I20260806 00:54:07.918787 2572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:07.943104 2572 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:54:07.944362 2572 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:07.947957 2572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bb581623dfb4b8b89f0467f57887e70: Generated new TSK 0
I20260806 00:54:07.948654 2572 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 00:54:07.988674 2594 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 00:54:07.988852 2594 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 00:54:07.989418 2557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:07.964417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53516 (local address 127.27.77.254:41881)
0806 00:54:07.964602 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:07.964609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:07.964828 (+ 219us) server_negotiation.cc:407] Connection header received
0806 00:54:07.965079 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:07.965081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:07.965131 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:07.965209 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0806 00:54:07.965531 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:07.966066 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:07.967334 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:07.967650 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:07.967735 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:07.968125 (+ 390us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 00:54:07.988860 (+ 20735us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 00:54:07.989255 (+ 395us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53516: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":91}
W20260806 00:54:07.989538 2557 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53516: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 00:54:07.990836 2593 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:07.964321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41881 (local address 127.0.0.1:53516)
0806 00:54:07.964730 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:07.964747 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:07.964892 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:07.965254 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:07.965257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:07.965266 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 00:54:07.965412 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:07.965419 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:07.966219 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:07.966222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:07.967201 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:07.967212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:07.967769 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:07.967773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:07.967869 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:07.967924 (+ 55us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 00:54:07.989176 (+ 21252us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 00:54:07.989471 (+ 295us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.77.254:41881: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":135,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.27.77.254:41881: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 00:54:07.991779 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 2465
2026-08-06T00:54:08Z chronyd exiting
I20260806 00:54:08.036542 27959 test_util.cc:175] -----------------------------------------------
I20260806 00:54:08.036639 27959 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977619051187-27959-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (349 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 00:54:08.037812 27959 mini_oidc.cc:152] Starting JWKS server
I20260806 00:54:08.038741 27959 webserver.cc:460] Webserver started at https://127.0.0.1:39327/ using document root <none> and password file <none>
I20260806 00:54:08.038859 27959 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 00:54:08.039148 27959 webserver.cc:460] Webserver started at http://127.0.0.1:33693/ using document root <none> and password file <none>
2026-08-06T00:54:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:08Z Disabled control of system clock
I20260806 00:54:08.049996 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:43781
--webserver_interface=127.27.77.254
--webserver_port=0
--builtin_ntp_servers=127.27.77.212:45817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:43781
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39327/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/testchainca.cert with env {}
W20260806 00:54:08.161973 2603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:08.162276 2603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:08.162356 2603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:08.166421 2603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:08.166515 2603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:08.166545 2603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:08.166587 2603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:08.166620 2603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 00:54:08.166658 2603 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 00:54:08.166687 2603 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39327/jwks.json
I20260806 00:54:08.172083 2603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:45817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:43781
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:43781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39327/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:08.173154 2603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:08.174183 2603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:08.182996 2614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:08.182981 2611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:08.183014 2608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:08.183207 2603 server_base.cc:1034] running on GCE node
I20260806 00:54:08.183909 2603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:08.185038 2603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:08.186236 2603 hybrid_clock.cc:648] HybridClock initialized: now 1785977648186218 us; error 33 us; skew 500 ppm
I20260806 00:54:08.188290 2603 webserver.cc:460] Webserver started at http://127.27.77.254:35929/ using document root <none> and password file <none>
I20260806 00:54:08.188903 2603 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:08.189002 2603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:08.189245 2603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:08.191344 2603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "485b40d9bfd3468dbe8e1c9bb6dc2522"
format_stamp: "Formatted at 2026-08-06 00:54:08 on dist-test-slave-sb2z"
server_key: "acbb4fbcea5e8d48c1ca7de2e785382f"
server_key_iv: "e73a191286e5d0b9a00ba9d11f243fa1"
server_key_version: "encryptionkey@0"
I20260806 00:54:08.191983 2603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "485b40d9bfd3468dbe8e1c9bb6dc2522"
format_stamp: "Formatted at 2026-08-06 00:54:08 on dist-test-slave-sb2z"
server_key: "acbb4fbcea5e8d48c1ca7de2e785382f"
server_key_iv: "e73a191286e5d0b9a00ba9d11f243fa1"
server_key_version: "encryptionkey@0"
I20260806 00:54:08.195822 2603 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:54:08.198199 2625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:08.199434 2603 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:54:08.199625 2603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "485b40d9bfd3468dbe8e1c9bb6dc2522"
format_stamp: "Formatted at 2026-08-06 00:54:08 on dist-test-slave-sb2z"
server_key: "acbb4fbcea5e8d48c1ca7de2e785382f"
server_key_iv: "e73a191286e5d0b9a00ba9d11f243fa1"
server_key_version: "encryptionkey@0"
I20260806 00:54:08.199746 2603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:08.218710 2603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:08.219470 2603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:08.219718 2603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:08.227274 2603 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:43781
I20260806 00:54:08.227294 2687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:43781 every 8 connection(s)
I20260806 00:54:08.228271 2603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:08.231417 2691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:08.236212 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 2603
I20260806 00:54:08.236351 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:54:08.236686 27959 external_mini_cluster.cc:1428] Setting key 86916596c074a762ebe057c8cdaf1205
I20260806 00:54:08.237493 2691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522: Bootstrap starting.
I20260806 00:54:08.240181 2691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:08.240940 2691 log.cc:824] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:08.242949 2691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522: No bootstrap required, opened a new log
I20260806 00:54:08.246706 2695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:08.238190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36770 (local address 127.27.77.254:43781)
0806 00:54:08.238875 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:08.238891 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:08.238927 (+ 36us) server_negotiation.cc:407] Connection header received
0806 00:54:08.239651 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:08.239669 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:08.240058 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:08.240386 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:08.240831 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.242305 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.243266 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.243803 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.243989 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:08.244905 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:08.244910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:08.244922 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:08.245320 (+ 398us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:08.245378 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:08.245398 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:08.245521 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:08.245673 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 00:54:08.245926 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":140,"threads_started":1}
I20260806 00:54:08.246913 2691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "485b40d9bfd3468dbe8e1c9bb6dc2522" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43781 } }
I20260806 00:54:08.247166 2691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:08.247248 2691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 485b40d9bfd3468dbe8e1c9bb6dc2522, State: Initialized, Role: FOLLOWER
I20260806 00:54:08.247771 2691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "485b40d9bfd3468dbe8e1c9bb6dc2522" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43781 } }
I20260806 00:54:08.247936 2691 raft_consensus.cc:388] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:08.248001 2691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:08.248107 2691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:08.249223 2691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "485b40d9bfd3468dbe8e1c9bb6dc2522" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43781 } }
I20260806 00:54:08.249588 2691 leader_election.cc:302] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 485b40d9bfd3468dbe8e1c9bb6dc2522; no voters:
I20260806 00:54:08.249788 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:54:08.249946 2691 leader_election.cc:288] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:54:08.250080 2697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:08.250345 2697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [term 1 LEADER]: Becoming Leader. State: Replica: 485b40d9bfd3468dbe8e1c9bb6dc2522, State: Running, Role: LEADER
I20260806 00:54:08.250730 2697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "485b40d9bfd3468dbe8e1c9bb6dc2522" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43781 } }
I20260806 00:54:08.251236 2691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:08.251991 2697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 485b40d9bfd3468dbe8e1c9bb6dc2522. Latest consensus state: current_term: 1 leader_uuid: "485b40d9bfd3468dbe8e1c9bb6dc2522" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "485b40d9bfd3468dbe8e1c9bb6dc2522" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43781 } } }
I20260806 00:54:08.252194 2697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:08.252411 2699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "485b40d9bfd3468dbe8e1c9bb6dc2522" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "485b40d9bfd3468dbe8e1c9bb6dc2522" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 43781 } } }
I20260806 00:54:08.252566 2699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:08.252684 2711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:08.255867 2711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:08.261992 2711 catalog_manager.cc:1347] Generated new cluster ID: 01d4bf35d8464cbb98e2a4f772b05e49
I20260806 00:54:08.262099 2711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:08.279311 2711 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:54:08.280700 2711 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:08.288647 2711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 485b40d9bfd3468dbe8e1c9bb6dc2522: Generated new TSK 0
I20260806 00:54:08.289391 2711 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 00:54:08.319466 2730 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 00:54:08.319658 2730 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 00:54:08.320374 2695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:08.303134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36778 (local address 127.27.77.254:43781)
0806 00:54:08.303369 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:08.303376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:08.303506 (+ 130us) server_negotiation.cc:407] Connection header received
0806 00:54:08.303767 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:08.303770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:08.303823 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:08.304022 (+ 199us) server_negotiation.cc:226] Negotiated authn=JWT
0806 00:54:08.304432 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.305055 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.306040 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.306297 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.306420 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:08.306779 (+ 359us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 00:54:08.319915 (+ 13136us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39327/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 00:54:08.320225 (+ 310us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36778: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39327/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 00:54:08.320465 2695 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36778: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39327/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 00:54:08.320497 2729 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:08.303055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:43781 (local address 127.0.0.1:36778)
0806 00:54:08.303410 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:08.303425 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:08.303610 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:08.304013 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:08.304017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:08.304026 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 00:54:08.304156 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:08.304167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.305191 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.305195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:08.305901 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:08.305909 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.306454 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.306460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:08.306559 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:08.306584 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 00:54:08.320131 (+ 13547us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39327/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 00:54:08.320303 (+ 172us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.77.254:43781: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39327/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":153,"threads_started":1}
I20260806 00:54:08.321187 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 2603
2026-08-06T00:54:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (329 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 00:54:08.367306 27959 mini_oidc.cc:152] Starting JWKS server
I20260806 00:54:08.368294 27959 webserver.cc:460] Webserver started at https://127.0.0.1:46201/ using document root <none> and password file <none>
I20260806 00:54:08.368436 27959 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 00:54:08.368706 27959 webserver.cc:460] Webserver started at http://127.0.0.1:42175/ using document root <none> and password file <none>
2026-08-06T00:54:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:08Z Disabled control of system clock
I20260806 00:54:08.379779 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:39505
--webserver_interface=127.27.77.254
--webserver_port=0
--builtin_ntp_servers=127.27.77.212:43925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:39505
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46201/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 00:54:08.494747 2738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:08.495136 2738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:08.495182 2738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:08.499150 2738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:08.499215 2738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:08.499231 2738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:08.499248 2738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:08.499264 2738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 00:54:08.499280 2738 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 00:54:08.499293 2738 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46201/jwks.json
I20260806 00:54:08.504019 2738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:39505
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:39505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46201/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:08.505143 2738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:08.506562 2738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:08.515353 2745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:08.515372 2748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:08.515353 2746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:08.515761 2738 server_base.cc:1034] running on GCE node
I20260806 00:54:08.516382 2738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:08.517522 2738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:08.518725 2738 hybrid_clock.cc:648] HybridClock initialized: now 1785977648518716 us; error 34 us; skew 500 ppm
I20260806 00:54:08.520707 2738 webserver.cc:460] Webserver started at http://127.27.77.254:35367/ using document root <none> and password file <none>
I20260806 00:54:08.521322 2738 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:08.521420 2738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:08.521677 2738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:08.523844 2738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "797d86c8871248a589f77a9f9b068579"
format_stamp: "Formatted at 2026-08-06 00:54:08 on dist-test-slave-sb2z"
server_key: "08414c179c045971a3815edfc10200de"
server_key_iv: "374d732077f6198c721fc9103ce86c16"
server_key_version: "encryptionkey@0"
I20260806 00:54:08.524418 2738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "797d86c8871248a589f77a9f9b068579"
format_stamp: "Formatted at 2026-08-06 00:54:08 on dist-test-slave-sb2z"
server_key: "08414c179c045971a3815edfc10200de"
server_key_iv: "374d732077f6198c721fc9103ce86c16"
server_key_version: "encryptionkey@0"
I20260806 00:54:08.528282 2738 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:54:08.530767 2758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:08.532311 2738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:54:08.532480 2738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "797d86c8871248a589f77a9f9b068579"
format_stamp: "Formatted at 2026-08-06 00:54:08 on dist-test-slave-sb2z"
server_key: "08414c179c045971a3815edfc10200de"
server_key_iv: "374d732077f6198c721fc9103ce86c16"
server_key_version: "encryptionkey@0"
I20260806 00:54:08.532614 2738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:08.544406 2738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:08.545153 2738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:08.545387 2738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:08.553536 2738 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:39505
I20260806 00:54:08.553552 2825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:39505 every 8 connection(s)
I20260806 00:54:08.554592 2738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:08.556123 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 2738
I20260806 00:54:08.556282 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:54:08.556571 27959 external_mini_cluster.cc:1428] Setting key 226b663db62e735b89ab74f5eb282af4
I20260806 00:54:08.558166 2826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:08.564652 2826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579: Bootstrap starting.
I20260806 00:54:08.566104 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:08.558003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58382 (local address 127.27.77.254:39505)
0806 00:54:08.558700 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:08.558716 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:08.558752 (+ 36us) server_negotiation.cc:407] Connection header received
0806 00:54:08.559379 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:08.559395 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:08.559863 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:08.560139 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:08.560547 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.561550 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.562816 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.563278 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.563399 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:08.564304 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:08.564309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:08.564322 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:08.564717 (+ 395us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:08.564802 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:08.564835 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:08.564924 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:08.565101 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 00:54:08.565394 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":157,"threads_started":1}
I20260806 00:54:08.567059 2826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:08.567862 2826 log.cc:824] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:08.568954 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:54:08.570222 2826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579: No bootstrap required, opened a new log
I20260806 00:54:08.573963 2826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797d86c8871248a589f77a9f9b068579" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39505 } }
I20260806 00:54:08.574185 2826 raft_consensus.cc:374] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:08.574249 2826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 797d86c8871248a589f77a9f9b068579, State: Initialized, Role: FOLLOWER
I20260806 00:54:08.574666 2826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797d86c8871248a589f77a9f9b068579" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39505 } }
I20260806 00:54:08.574813 2826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:08.574904 2826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:08.575022 2826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:08.576031 2826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797d86c8871248a589f77a9f9b068579" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39505 } }
I20260806 00:54:08.576392 2826 leader_election.cc:302] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 797d86c8871248a589f77a9f9b068579; no voters:
I20260806 00:54:08.576711 2826 leader_election.cc:288] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:54:08.576936 2831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:08.577167 2831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [term 1 LEADER]: Becoming Leader. State: Replica: 797d86c8871248a589f77a9f9b068579, State: Running, Role: LEADER
I20260806 00:54:08.577466 2831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797d86c8871248a589f77a9f9b068579" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39505 } }
I20260806 00:54:08.578054 2826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:08.579226 2834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 797d86c8871248a589f77a9f9b068579. Latest consensus state: current_term: 1 leader_uuid: "797d86c8871248a589f77a9f9b068579" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797d86c8871248a589f77a9f9b068579" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39505 } } }
I20260806 00:54:08.579385 2834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:08.579618 2833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "797d86c8871248a589f77a9f9b068579" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797d86c8871248a589f77a9f9b068579" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39505 } } }
I20260806 00:54:08.579767 2833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:08.579844 2843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:08.583081 2843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:08.588863 2843 catalog_manager.cc:1347] Generated new cluster ID: cd0064c6ca3b49ea86b46459429b22b2
I20260806 00:54:08.588932 2843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:08.597121 2843 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:54:08.597986 2843 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:08.603561 2843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 797d86c8871248a589f77a9f9b068579: Generated new TSK 0
I20260806 00:54:08.604318 2843 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:54:08.622777 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:08.617592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58398 (local address 127.27.77.254:39505)
0806 00:54:08.617740 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:08.617745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:08.617925 (+ 180us) server_negotiation.cc:407] Connection header received
0806 00:54:08.618117 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:08.618119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:08.618167 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:08.618257 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0806 00:54:08.618549 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.619053 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.620501 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.620715 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.620789 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:08.621158 (+ 369us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 00:54:08.622379 (+ 1221us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 00:54:08.622633 (+ 254us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58398: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":52}
W20260806 00:54:08.622869 2863 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:08.617461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:39505 (local address 127.0.0.1:58398)
0806 00:54:08.617817 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:08.617834 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:08.617980 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:08.618288 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:08.618292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:08.618301 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 00:54:08.618434 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:08.618441 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.619193 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.619197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:08.620354 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:08.620366 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.620837 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.620841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:08.620959 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:08.620989 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 00:54:08.622551 (+ 1562us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 00:54:08.622696 (+ 145us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.77.254:39505: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":119,"threads_started":1}
W20260806 00:54:08.623225 2829 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58398: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260806 00:54:08.623471 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 2738
E20260806 00:54:08.642048 2866 webserver.cc:533] Webserver: sslize failed:
E20260806 00:54:08.642163 2866 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T00:54:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (303 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T00:54:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:08Z Disabled control of system clock
I20260806 00:54:08.681123 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:35953
--webserver_interface=127.27.77.254
--webserver_port=0
--builtin_ntp_servers=127.27.77.212:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:35953
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/testchainca.cert with env {}
W20260806 00:54:08.796307 2871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:08.796613 2871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:08.796692 2871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:08.800701 2871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:08.800791 2871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:08.800822 2871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:08.800859 2871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:08.800892 2871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 00:54:08.800958 2871 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 00:54:08.800992 2871 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 00:54:08.806331 2871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:35953
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:35953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:08.807408 2871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:08.808470 2871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:08.816969 2880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:08.816994 2878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:08.817000 2877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:08.817790 2871 server_base.cc:1034] running on GCE node
I20260806 00:54:08.818365 2871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:08.819516 2871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:08.820796 2871 hybrid_clock.cc:648] HybridClock initialized: now 1785977648820768 us; error 45 us; skew 500 ppm
I20260806 00:54:08.827437 2871 webserver.cc:460] Webserver started at http://127.27.77.254:37705/ using document root <none> and password file <none>
I20260806 00:54:08.828097 2871 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:08.828199 2871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:08.828433 2871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:08.830492 2871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "5506b9823e1344a19af41751ba6f9422"
format_stamp: "Formatted at 2026-08-06 00:54:08 on dist-test-slave-sb2z"
server_key: "bf04aadf65ed422e5cfad2bdceede6b8"
server_key_iv: "0b491d401265c190611892fefe4b72a0"
server_key_version: "encryptionkey@0"
I20260806 00:54:08.831075 2871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "5506b9823e1344a19af41751ba6f9422"
format_stamp: "Formatted at 2026-08-06 00:54:08 on dist-test-slave-sb2z"
server_key: "bf04aadf65ed422e5cfad2bdceede6b8"
server_key_iv: "0b491d401265c190611892fefe4b72a0"
server_key_version: "encryptionkey@0"
I20260806 00:54:08.834810 2871 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260806 00:54:08.837409 2887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:08.838688 2871 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:54:08.838842 2871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "5506b9823e1344a19af41751ba6f9422"
format_stamp: "Formatted at 2026-08-06 00:54:08 on dist-test-slave-sb2z"
server_key: "bf04aadf65ed422e5cfad2bdceede6b8"
server_key_iv: "0b491d401265c190611892fefe4b72a0"
server_key_version: "encryptionkey@0"
I20260806 00:54:08.838976 2871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:08.858454 2871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:08.859249 2871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:08.859473 2871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:08.866842 2871 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:35953
I20260806 00:54:08.866865 2957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:35953 every 8 connection(s)
I20260806 00:54:08.867900 2871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:08.870847 2958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:08.876543 2958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422: Bootstrap starting.
I20260806 00:54:08.877436 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 2871
I20260806 00:54:08.877599 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:54:08.877876 27959 external_mini_cluster.cc:1428] Setting key 952e80f54fc7680476d0f897e4c7cc92
I20260806 00:54:08.879493 2958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:08.880281 2958 log.cc:824] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:08.882268 2958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422: No bootstrap required, opened a new log
I20260806 00:54:08.885772 2958 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5506b9823e1344a19af41751ba6f9422" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 35953 } }
I20260806 00:54:08.886018 2958 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:08.886081 2958 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5506b9823e1344a19af41751ba6f9422, State: Initialized, Role: FOLLOWER
I20260806 00:54:08.886565 2958 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5506b9823e1344a19af41751ba6f9422" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 35953 } }
I20260806 00:54:08.886725 2958 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:08.886801 2958 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:08.886914 2958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:08.887467 2962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:08.878999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47320 (local address 127.27.77.254:35953)
0806 00:54:08.879813 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:08.879829 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:08.879867 (+ 38us) server_negotiation.cc:407] Connection header received
0806 00:54:08.880634 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:08.880654 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:08.881007 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:08.881360 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:08.881823 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.882866 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.883849 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.884321 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.884502 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:08.885385 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:08.885390 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:08.885403 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:08.885804 (+ 401us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:08.885858 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:08.885877 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:08.885987 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:08.886179 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 00:54:08.886461 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":129,"threads_started":1}
I20260806 00:54:08.888098 2958 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5506b9823e1344a19af41751ba6f9422" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 35953 } }
I20260806 00:54:08.888443 2958 leader_election.cc:302] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5506b9823e1344a19af41751ba6f9422; no voters:
I20260806 00:54:08.888818 2958 leader_election.cc:288] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:54:08.888900 2964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:08.889083 2964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [term 1 LEADER]: Becoming Leader. State: Replica: 5506b9823e1344a19af41751ba6f9422, State: Running, Role: LEADER
I20260806 00:54:08.889376 2964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5506b9823e1344a19af41751ba6f9422" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 35953 } }
I20260806 00:54:08.890105 2958 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:08.890591 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:54:08.891371 2965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5506b9823e1344a19af41751ba6f9422" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5506b9823e1344a19af41751ba6f9422" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 35953 } } }
I20260806 00:54:08.891502 2965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:08.891956 2976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:08.891889 2967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5506b9823e1344a19af41751ba6f9422. Latest consensus state: current_term: 1 leader_uuid: "5506b9823e1344a19af41751ba6f9422" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5506b9823e1344a19af41751ba6f9422" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 35953 } } }
I20260806 00:54:08.892048 2967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:08.895057 2976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:08.900784 2976 catalog_manager.cc:1347] Generated new cluster ID: cd38906b7d0049aa902f553852ed3d4a
I20260806 00:54:08.900864 2976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:08.913843 2976 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:54:08.914844 2976 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:08.923329 2976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5506b9823e1344a19af41751ba6f9422: Generated new TSK 0
I20260806 00:54:08.924047 2976 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:54:09.391383 2962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:08.935416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47322 (local address 127.27.77.254:35953)
0806 00:54:08.935567 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:08.935572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:08.935726 (+ 154us) server_negotiation.cc:407] Connection header received
0806 00:54:08.935863 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:08.935868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:08.935926 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:08.936054 (+ 128us) server_negotiation.cc:226] Negotiated authn=JWT
0806 00:54:08.936323 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.937166 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.938092 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.938447 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.938555 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:08.938879 (+ 324us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 00:54:09.390752 (+451873us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 00:54:09.391246 (+ 494us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47322: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 00:54:09.391496 2962 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47322: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 00:54:09.391633 2995 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:08.935307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:35953 (local address 127.0.0.1:47322)
0806 00:54:08.935640 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:08.935653 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:08.935750 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:08.936079 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:08.936082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:08.936101 (+ 19us) client_negotiation.cc:189] Negotiated authn=JWT
0806 00:54:08.936207 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:08.936212 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.937318 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.937329 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:08.937972 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:08.937980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:08.938593 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:08.938597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:08.938669 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:08.938693 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 00:54:09.391163 (+452470us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 00:54:09.391399 (+ 236us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.77.254:35953: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":130,"threads_started":1}
I20260806 00:54:09.392398 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 2871
2026-08-06T00:54:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (747 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (132 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (128 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:54:09 dist-test-slave-sb2z krb5kdc[3009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:54:09 dist-test-slave-sb2z krb5kdc[3009](info): set up 2 sockets
Aug 06 00:54:09 dist-test-slave-sb2z krb5kdc[3009](info): commencing operation
krb5kdc: starting...
W20260806 00:54:10.856379 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:54:10 dist-test-slave-sb2z krb5kdc[3009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:54:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:54:10.991912 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:33983
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:33983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:11.107785 3025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:11.108063 3025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:11.108104 3025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:11.112071 3025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:11.112134 3025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:11.112154 3025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:11.112171 3025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:11.112187 3025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:54:11.117410 3025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:33983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:33983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:11.118451 3025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:11.119441 3025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:11.128228 3030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:11.128290 3034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:11.128379 3025 server_base.cc:1034] running on GCE node
W20260806 00:54:11.128253 3032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:11.129226 3025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:11.130309 3025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:11.131488 3025 hybrid_clock.cc:648] HybridClock initialized: now 1785977651131468 us; error 32 us; skew 500 ppm
Aug 06 00:54:11 dist-test-slave-sb2z krb5kdc[3009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977651, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:11.134547 3025 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:54:11.135684 3025 webserver.cc:460] Webserver started at http://127.27.77.254:46219/ using document root <none> and password file <none>
I20260806 00:54:11.136270 3025 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:11.136335 3025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:11.136636 3025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:11.138773 3025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "ce20e00a62d443609c9a9a7c21ad4501"
format_stamp: "Formatted at 2026-08-06 00:54:11 on dist-test-slave-sb2z"
server_key: "bbda089e537c6e5441a49277a79af6ac"
server_key_iv: "7b53da919207afd8e7bf812ea0f59b84"
server_key_version: "encryptionkey@0"
I20260806 00:54:11.139389 3025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "ce20e00a62d443609c9a9a7c21ad4501"
format_stamp: "Formatted at 2026-08-06 00:54:11 on dist-test-slave-sb2z"
server_key: "bbda089e537c6e5441a49277a79af6ac"
server_key_iv: "7b53da919207afd8e7bf812ea0f59b84"
server_key_version: "encryptionkey@0"
I20260806 00:54:11.143306 3025 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260806 00:54:11.145901 3041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:11.147185 3025 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:54:11.147325 3025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "ce20e00a62d443609c9a9a7c21ad4501"
format_stamp: "Formatted at 2026-08-06 00:54:11 on dist-test-slave-sb2z"
server_key: "bbda089e537c6e5441a49277a79af6ac"
server_key_iv: "7b53da919207afd8e7bf812ea0f59b84"
server_key_version: "encryptionkey@0"
I20260806 00:54:11.147437 3025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:11.171440 3025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:11.175145 3025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:11.175387 3025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:11.183053 3025 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:33983
I20260806 00:54:11.183079 3103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:33983 every 8 connection(s)
I20260806 00:54:11.184146 3025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:11.187251 3104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:11.189505 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 3025
I20260806 00:54:11.189620 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:54:11.189945 27959 external_mini_cluster.cc:1428] Setting key 91f022b47956447e6b8eb85d8db0dc86
I20260806 00:54:11.193986 3104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501: Bootstrap starting.
I20260806 00:54:11.196528 3104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:11.197388 3104 log.cc:824] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:54:11 dist-test-slave-sb2z krb5kdc[3009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:11.200006 3104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501: No bootstrap required, opened a new log
I20260806 00:54:11.203900 3104 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce20e00a62d443609c9a9a7c21ad4501" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33983 } }
I20260806 00:54:11.204206 3104 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:11.204314 3104 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce20e00a62d443609c9a9a7c21ad4501, State: Initialized, Role: FOLLOWER
I20260806 00:54:11.204878 3104 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce20e00a62d443609c9a9a7c21ad4501" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33983 } }
I20260806 00:54:11.205031 3104 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:11.205138 3104 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:11.205298 3104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:11.205463 3107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:11.191512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.27.77.254:33983)
0806 00:54:11.192011 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:11.192026 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:11.192063 (+ 37us) server_negotiation.cc:407] Connection header received
0806 00:54:11.192977 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:11.192996 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:11.193371 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:11.193724 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:11.194598 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:11.195652 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:11.196513 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:11.197065 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:11.197253 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:11.200370 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:11.200375 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:11.200389 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:11.200418 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:11.202254 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:11.202743 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:11.202748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:11.202753 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:11.202869 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:11.203178 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:11.203182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:11.203184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:11.203671 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:11.203894 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:11.204184 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 00:54:11.204520 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":150,"threads_started":1}
I20260806 00:54:11.206451 3104 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce20e00a62d443609c9a9a7c21ad4501" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33983 } }
I20260806 00:54:11.206864 3104 leader_election.cc:302] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce20e00a62d443609c9a9a7c21ad4501; no voters:
I20260806 00:54:11.207266 3104 leader_election.cc:288] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:54:11.207381 3109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:11.207674 3109 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [term 1 LEADER]: Becoming Leader. State: Replica: ce20e00a62d443609c9a9a7c21ad4501, State: Running, Role: LEADER
I20260806 00:54:11.208039 3109 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce20e00a62d443609c9a9a7c21ad4501" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33983 } }
I20260806 00:54:11.208696 3104 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:11.209206 3109 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce20e00a62d443609c9a9a7c21ad4501. Latest consensus state: current_term: 1 leader_uuid: "ce20e00a62d443609c9a9a7c21ad4501" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce20e00a62d443609c9a9a7c21ad4501" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33983 } } }
I20260806 00:54:11.209376 3109 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:11.209746 3110 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce20e00a62d443609c9a9a7c21ad4501" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce20e00a62d443609c9a9a7c21ad4501" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33983 } } }
I20260806 00:54:11.209935 3110 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:11.210253 3114 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:11.213927 3114 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:11.219960 3114 catalog_manager.cc:1347] Generated new cluster ID: ac2d5b66703e4c2faca336f97b2d2bf6
I20260806 00:54:11.220119 3114 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:11.229789 3114 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:54:11.230785 3114 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:11.241988 3114 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce20e00a62d443609c9a9a7c21ad4501: Generated new TSK 0
I20260806 00:54:11.242779 3114 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:54:11.278908 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33983
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:11.398859 3139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:11.399135 3139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:11.399179 3139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:11.403177 3139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:11.403241 3139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:11.403299 3139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:54:11.408497 3139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:11.409740 3139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:11.410861 3139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:11.420797 3147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:11.420770 3148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:11.420856 3152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:11.421139 3139 server_base.cc:1034] running on GCE node
I20260806 00:54:11.421588 3139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:11.422245 3139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:11.423413 3139 hybrid_clock.cc:648] HybridClock initialized: now 1785977651423391 us; error 32 us; skew 500 ppm
Aug 06 00:54:11 dist-test-slave-sb2z krb5kdc[3009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977651, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:11.426393 3139 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:54:11.427294 3139 webserver.cc:460] Webserver started at http://127.27.77.193:35201/ using document root <none> and password file <none>
I20260806 00:54:11.428010 3139 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:11.428117 3139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:11.428359 3139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:11.430384 3139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "c96760fd0f8c48858c9aa74469730ecb"
format_stamp: "Formatted at 2026-08-06 00:54:11 on dist-test-slave-sb2z"
server_key: "9717a8aedcdf643e8cf6d069ff4fb25d"
server_key_iv: "76a659df9fec33d74d6daa2d8027ebbd"
server_key_version: "encryptionkey@0"
I20260806 00:54:11.430956 3139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "c96760fd0f8c48858c9aa74469730ecb"
format_stamp: "Formatted at 2026-08-06 00:54:11 on dist-test-slave-sb2z"
server_key: "9717a8aedcdf643e8cf6d069ff4fb25d"
server_key_iv: "76a659df9fec33d74d6daa2d8027ebbd"
server_key_version: "encryptionkey@0"
I20260806 00:54:11.434790 3139 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:54:11.437373 3162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:11.438565 3139 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:54:11.438709 3139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "c96760fd0f8c48858c9aa74469730ecb"
format_stamp: "Formatted at 2026-08-06 00:54:11 on dist-test-slave-sb2z"
server_key: "9717a8aedcdf643e8cf6d069ff4fb25d"
server_key_iv: "76a659df9fec33d74d6daa2d8027ebbd"
server_key_version: "encryptionkey@0"
I20260806 00:54:11.438822 3139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:11.466101 3139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:11.469759 3139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:11.469933 3139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:11.470523 3139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:11.471351 3139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:11.471400 3139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:11.471441 3139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:11.471467 3139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:11.481442 3139 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:35171
I20260806 00:54:11.481477 3321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:35171 every 8 connection(s)
I20260806 00:54:11.482410 3139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:54:11.486836 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 3139
I20260806 00:54:11.486976 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:54:11.487370 27959 external_mini_cluster.cc:1428] Setting key bd3d8284f6f54e14a6dcfa43d5659877
Aug 06 00:54:11 dist-test-slave-sb2z krb5kdc[3009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977651, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:11.498504 3107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:11.484499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:41481 (local address 127.27.77.254:33983)
0806 00:54:11.484652 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:11.484655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:11.485499 (+ 844us) server_negotiation.cc:407] Connection header received
0806 00:54:11.487173 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:11.487176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:11.487229 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:11.487307 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:11.488783 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:11.489378 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:11.491223 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:11.491673 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:11.491805 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:11.494387 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:11.494392 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:11.494396 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:11.494431 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:11.496236 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:11.496784 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:11.496789 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:11.496791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:11.496898 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:11.497253 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:11.497258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:11.497261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:11.497544 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:11.497655 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:11.498175 (+ 520us) server_negotiation.cc:299] Negotiation successful
0806 00:54:11.498326 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
I20260806 00:54:11.499328 3327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:11.484864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:33983 (local address 127.27.77.193:41481)
0806 00:54:11.485344 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:11.485387 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:11.486869 (+ 1482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:11.487542 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:11.487553 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:11.488060 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:11.488621 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:11.488643 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:11.489523 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:11.489530 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:11.491047 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:11.491064 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:11.491833 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:11.491836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:11.491981 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:11.492622 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:11.492639 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:11.494199 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:11.496362 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:11.496367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:11.496382 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:11.496669 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:11.497003 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:11.497006 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:11.497007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:11.497149 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:11.497659 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:11.497664 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:11.497911 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 00:54:11.498074 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 00:54:11.498364 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":126,"threads_started":1}
I20260806 00:54:11.501003 3324 heartbeater.cc:347] Connected to a master server at 127.27.77.254:33983
I20260806 00:54:11.501271 3324 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
I20260806 00:54:11.501927 3324 heartbeater.cc:511] Master 127.27.77.254:33983 requested a full tablet report, sending...
I20260806 00:54:11.503625 3062 ts_manager.cc:194] Registered new tserver with Master: c96760fd0f8c48858c9aa74469730ecb (127.27.77.193:35171)
I20260806 00:54:11.504805 3062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:41481
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:54:11.519872 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33983
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:11.637305 3330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:11.637756 3330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:11.637848 3330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:11.642655 3330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:11.642758 3330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:11.642845 3330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:54:11.649026 3330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:11.650286 3330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:11.651401 3330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:54:11.661823 3330 server_base.cc:1034] running on GCE node
W20260806 00:54:11.661790 3336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:11.661779 3335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:11.661779 3338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:11.662432 3330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:11.663076 3330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:11.664273 3330 hybrid_clock.cc:648] HybridClock initialized: now 1785977651664259 us; error 26 us; skew 500 ppm
Aug 06 00:54:11 dist-test-slave-sb2z krb5kdc[3009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977651, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:11.667049 3330 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:54:11.668210 3330 webserver.cc:460] Webserver started at http://127.27.77.194:33505/ using document root <none> and password file <none>
I20260806 00:54:11.668835 3330 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:11.668937 3330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:11.669191 3330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:11.671358 3330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "2ba54e8003c74630acc1eb616e70ca6b"
format_stamp: "Formatted at 2026-08-06 00:54:11 on dist-test-slave-sb2z"
server_key: "864b7de017037e78033f97c883453dde"
server_key_iv: "56eb5b795679f7c9041a02edb0b3a25c"
server_key_version: "encryptionkey@0"
I20260806 00:54:11.672000 3330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "2ba54e8003c74630acc1eb616e70ca6b"
format_stamp: "Formatted at 2026-08-06 00:54:11 on dist-test-slave-sb2z"
server_key: "864b7de017037e78033f97c883453dde"
server_key_iv: "56eb5b795679f7c9041a02edb0b3a25c"
server_key_version: "encryptionkey@0"
I20260806 00:54:11.675849 3330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:54:11.678689 3346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:11.680675 3330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:54:11.680827 3330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "2ba54e8003c74630acc1eb616e70ca6b"
format_stamp: "Formatted at 2026-08-06 00:54:11 on dist-test-slave-sb2z"
server_key: "864b7de017037e78033f97c883453dde"
server_key_iv: "56eb5b795679f7c9041a02edb0b3a25c"
server_key_version: "encryptionkey@0"
I20260806 00:54:11.681000 3330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:11.698343 3330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:11.701902 3330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:11.702152 3330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:11.702821 3330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:11.703840 3330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:11.703920 3330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:11.704010 3330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:11.704070 3330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:11.714812 3506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:40609 every 8 connection(s)
I20260806 00:54:11.714807 3330 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:40609
I20260806 00:54:11.715924 3330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:54:11.717504 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 3330
I20260806 00:54:11.717648 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:54:11.718034 27959 external_mini_cluster.cc:1428] Setting key ac6157ca3d2954522915bde2a96f17f4
Aug 06 00:54:11 dist-test-slave-sb2z krb5kdc[3009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977651, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
I20260806 00:54:11.735795 3107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:11.718114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:57179 (local address 127.27.77.254:33983)
0806 00:54:11.718275 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:11.718279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:11.720276 (+ 1997us) server_negotiation.cc:407] Connection header received
0806 00:54:11.722678 (+ 2402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:11.722685 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:11.722758 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:11.722870 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:11.724449 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:11.725506 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:11.727694 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:11.727942 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:11.728040 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:11.731517 (+ 3477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:11.731566 (+ 49us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:11.731573 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:11.731613 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:11.733166 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:11.733832 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:11.733836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:11.733838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:11.733959 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:11.734469 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:11.734476 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:11.734479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:11.734698 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:11.734825 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:11.735414 (+ 589us) server_negotiation.cc:299] Negotiation successful
0806 00:54:11.735596 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 00:54:11.737087 3509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:11.718694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:33983 (local address 127.27.77.194:57179)
0806 00:54:11.720079 (+ 1385us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:11.720141 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:11.722357 (+ 2216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:11.723048 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:11.723055 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:11.723477 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:11.724188 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:11.724213 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:11.725726 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:11.725734 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:11.727552 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:11.727569 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:11.728136 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:11.728145 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:11.728414 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:11.729167 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:11.729193 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:11.731281 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:11.733341 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:11.733347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:11.733361 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:11.733683 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:11.734146 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:11.734154 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:11.734157 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:11.734330 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:11.734824 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:11.734834 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:11.735127 (+ 293us) client_negotiation.cc:769] Sending connection context
0806 00:54:11.735340 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 00:54:11.735751 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1140,"thread_start_us":165,"threads_started":1}
I20260806 00:54:11.738538 3507 heartbeater.cc:347] Connected to a master server at 127.27.77.254:33983
I20260806 00:54:11.738812 3507 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:11.739377 3507 heartbeater.cc:511] Master 127.27.77.254:33983 requested a full tablet report, sending...
I20260806 00:54:11.740531 3062 ts_manager.cc:194] Registered new tserver with Master: 2ba54e8003c74630acc1eb616e70ca6b (127.27.77.194:40609)
I20260806 00:54:11.741161 3062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:57179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:54:11.755218 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33983
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:11.882833 3512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:11.883122 3512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:11.883170 3512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:11.888240 3512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:11.888311 3512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:11.888409 3512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:54:11.893905 3512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:11.894999 3512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:11.896068 3512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:11.905974 3518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:11.905907 3517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:11.905982 3512 server_base.cc:1034] running on GCE node
W20260806 00:54:11.905907 3520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:11.906765 3512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:11.907380 3512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:11.908524 3512 hybrid_clock.cc:648] HybridClock initialized: now 1785977651908490 us; error 48 us; skew 500 ppm
Aug 06 00:54:11 dist-test-slave-sb2z krb5kdc[3009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977651, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:11.911720 3512 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:54:11.912678 3512 webserver.cc:460] Webserver started at http://127.27.77.195:45105/ using document root <none> and password file <none>
I20260806 00:54:11.913354 3512 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:11.913455 3512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:11.913705 3512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:11.915871 3512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "ff037970445a4394b885a6cf1557f7de"
format_stamp: "Formatted at 2026-08-06 00:54:11 on dist-test-slave-sb2z"
server_key: "ff8858eb8b950146d1b7f32f19b6ad57"
server_key_iv: "56b6be29dc2828b6994cf52daa29476e"
server_key_version: "encryptionkey@0"
I20260806 00:54:11.916492 3512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "ff037970445a4394b885a6cf1557f7de"
format_stamp: "Formatted at 2026-08-06 00:54:11 on dist-test-slave-sb2z"
server_key: "ff8858eb8b950146d1b7f32f19b6ad57"
server_key_iv: "56b6be29dc2828b6994cf52daa29476e"
server_key_version: "encryptionkey@0"
I20260806 00:54:11.920364 3512 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:54:11.922989 3530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:11.924374 3512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:54:11.924513 3512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "ff037970445a4394b885a6cf1557f7de"
format_stamp: "Formatted at 2026-08-06 00:54:11 on dist-test-slave-sb2z"
server_key: "ff8858eb8b950146d1b7f32f19b6ad57"
server_key_iv: "56b6be29dc2828b6994cf52daa29476e"
server_key_version: "encryptionkey@0"
I20260806 00:54:11.924623 3512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:11.944024 3512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:11.947281 3512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:11.947499 3512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:11.948203 3512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:11.949077 3512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:11.949126 3512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:11.949191 3512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:11.949224 3512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:11.958447 3512 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:41493
I20260806 00:54:11.958469 3691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:41493 every 8 connection(s)
I20260806 00:54:11.959362 3512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:54:11.963846 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 3512
I20260806 00:54:11.964010 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:54:11.964335 27959 external_mini_cluster.cc:1428] Setting key d5a272c1a1bf2b6cfb9dd905339c877d
Aug 06 00:54:11 dist-test-slave-sb2z krb5kdc[3009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977651, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:11.973572 3107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:11.961284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:37023 (local address 127.27.77.254:33983)
0806 00:54:11.961435 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:11.961439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:11.962339 (+ 900us) server_negotiation.cc:407] Connection header received
0806 00:54:11.963843 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:11.963846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:11.963896 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:11.963998 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:11.965395 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:11.965940 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:11.967184 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:11.967413 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:11.967527 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:11.970039 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:11.970043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:11.970046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:11.970080 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:11.971412 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:11.971995 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:11.971998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:11.972000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:11.972087 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:11.972454 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:11.972458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:11.972459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:11.972641 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:11.972752 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:11.973255 (+ 503us) server_negotiation.cc:299] Negotiation successful
0806 00:54:11.973384 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:54:11.974335 3695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:11.961660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:33983 (local address 127.27.77.195:37023)
0806 00:54:11.962180 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:11.962223 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:11.963587 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:11.964203 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:11.964213 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:11.964747 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:11.965241 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:11.965254 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:11.966081 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:11.966084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:11.967054 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:11.967064 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:11.967612 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:11.967617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:11.967780 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:11.968334 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:11.968351 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:11.969864 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:11.971577 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:11.971584 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:11.971599 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:11.971870 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:11.972184 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:11.972188 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:11.972190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:11.972340 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:11.972752 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:11.972760 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:11.973015 (+ 255us) client_negotiation.cc:769] Sending connection context
0806 00:54:11.973198 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 00:54:11.973523 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":113,"threads_started":1}
I20260806 00:54:11.975565 3692 heartbeater.cc:347] Connected to a master server at 127.27.77.254:33983
I20260806 00:54:11.975829 3692 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:11.976342 3692 heartbeater.cc:511] Master 127.27.77.254:33983 requested a full tablet report, sending...
I20260806 00:54:11.977507 3062 ts_manager.cc:194] Registered new tserver with Master: ff037970445a4394b885a6cf1557f7de (127.27.77.195:41493)
I20260806 00:54:11.978288 3062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:37023
I20260806 00:54:11.978497 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/test-admin.keytab.
Aug 06 00:54:11 dist-test-slave-sb2z krb5kdc[3009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:11.996819 27959 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 00:54:11.999871 3702 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:11.998332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57938 (local address 127.27.77.193:35171)
0806 00:54:11.998666 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:11.998682 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:11.998707 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:54:11.998760 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:11.998764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:11.998944 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:11.999064 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:11.999692 (+ 628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57938: BlockingRecv error: recv got EOF from 127.0.0.1:57938 (error 108)
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":129,"threads_started":1}
W20260806 00:54:12.000051 27959 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/krb5cc' of type FILE
Aug 06 00:54:11 dist-test-slave-sb2z krb5kdc[3009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:12.001134 27959 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 00:54:11 dist-test-slave-sb2z krb5kdc[3009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:12.002058 27959 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 00:54:12 dist-test-slave-sb2z krb5kdc[3009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.193@KRBTEST.COM
I20260806 00:54:12.013054 3702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:12.002967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57944 (local address 127.27.77.193:35171)
0806 00:54:12.003122 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:12.003125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:12.003140 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:54:12.003290 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:12.003292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:12.003337 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:12.003409 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:12.004084 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:12.004831 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:12.005735 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:12.006151 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:12.006238 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:12.008949 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:12.008961 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:12.008977 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:12.009004 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:12.011098 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:12.011617 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:12.011623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:12.011624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:12.011730 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:12.012028 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:12.012031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:12.012033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:12.012315 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:12.012549 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:12.012737 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 00:54:12.012896 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 00:54:12.017568 3702 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:12.016400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57952 (local address 127.27.77.193:35171)
0806 00:54:12.016557 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:12.016560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:12.016575 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:54:12.016856 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:12.016859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:12.016902 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:12.016977 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:12.017427 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57952: BlockingRecv error: recv got EOF from 127.0.0.1:57952 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 00:54:12.017704 27959 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/krb5cc' of type FILE
Aug 06 00:54:12 dist-test-slave-sb2z krb5kdc[3009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:12.018878 27959 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 00:54:12 dist-test-slave-sb2z krb5kdc[3009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:12.019816 27959 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 00:54:12 dist-test-slave-sb2z krb5kdc[3009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.193@KRBTEST.COM
I20260806 00:54:12.028421 3702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:12.020739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57958 (local address 127.27.77.193:35171)
0806 00:54:12.020869 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:12.020872 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:12.020886 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:54:12.021014 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:12.021016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:12.021052 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:12.021119 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:12.021615 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:12.022129 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:12.022876 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:12.023116 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:12.023202 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:12.025368 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:12.025371 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:12.025373 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:12.025396 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:12.026842 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:12.027210 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:12.027212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:12.027214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:12.027285 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:12.027506 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:12.027509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:12.027510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:12.027750 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:12.027855 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:12.028125 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 00:54:12.028282 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 00:54:12.031400 3702 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:12.030332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57966 (local address 127.27.77.193:35171)
0806 00:54:12.030469 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:12.030473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:12.030591 (+ 118us) server_negotiation.cc:407] Connection header received
0806 00:54:12.030740 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:12.030743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:12.030792 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:12.030872 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:12.031275 (+ 403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57966: BlockingRecv error: recv got EOF from 127.0.0.1:57966 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 00:54:12.031560 27959 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977619051187-27959-0/krb5kdc/krb5cc' of type FILE
Aug 06 00:54:12 dist-test-slave-sb2z krb5kdc[3009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:12.032569 27959 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 00:54:12 dist-test-slave-sb2z krb5kdc[3009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:12.033429 27959 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 00:54:12 dist-test-slave-sb2z krb5kdc[3009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.193@KRBTEST.COM
I20260806 00:54:12.042775 3702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:12.034363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57972 (local address 127.27.77.193:35171)
0806 00:54:12.034494 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:12.034497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:12.034580 (+ 83us) server_negotiation.cc:407] Connection header received
0806 00:54:12.034659 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:12.034661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:12.034698 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:12.034765 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:12.035367 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:12.035980 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:12.036784 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:12.036998 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:12.037091 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:12.039688 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:12.039692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:12.039694 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:12.039721 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:12.041137 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:12.041551 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:12.041554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:12.041555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:12.041645 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:12.041887 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:12.041890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:12.041891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:12.042062 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:12.042194 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:12.042456 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 00:54:12.042627 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 00:54:12.044059 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 3139
I20260806 00:54:12.057265 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 3330
I20260806 00:54:12.068341 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 3512
I20260806 00:54:12.085857 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 3025
2026-08-06T00:54:12Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2434 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:54:12 dist-test-slave-sb2z krb5kdc[3718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:54:12 dist-test-slave-sb2z krb5kdc[3718](info): set up 2 sockets
Aug 06 00:54:12 dist-test-slave-sb2z krb5kdc[3718](info): commencing operation
krb5kdc: starting...
W20260806 00:54:13.314857 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:54:13 dist-test-slave-sb2z krb5kdc[3718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:54:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:13Z Disabled control of system clock
WARNING: no policy specified for oryx/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.254.keytab.
Entry for principal oryx/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.254.keytab.
I20260806 00:54:13.459569 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38937
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.254.keytab
--principal=oryx/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:38937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:13.577924 3738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:13.578212 3738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:13.578254 3738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:13.582329 3738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:13.582401 3738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:13.582419 3738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:13.582436 3738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:13.582451 3738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:54:13.587924 3738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:38937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.254.keytab
--principal=oryx/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:13.589023 3738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:13.590086 3738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:13.599184 3744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:13.599265 3748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:13.599268 3738 server_base.cc:1034] running on GCE node
W20260806 00:54:13.599184 3745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:13.600338 3738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:13.601476 3738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:13.602697 3738 hybrid_clock.cc:648] HybridClock initialized: now 1785977653602666 us; error 46 us; skew 500 ppm
Aug 06 00:54:13 dist-test-slave-sb2z krb5kdc[3718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977653, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:13.605803 3738 init.cc:377] Logged in from keytab as oryx/127.27.77.254@KRBTEST.COM (short username oryx)
I20260806 00:54:13.606730 3738 webserver.cc:460] Webserver started at http://127.27.77.254:41815/ using document root <none> and password file <none>
I20260806 00:54:13.607295 3738 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:13.607354 3738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:13.607579 3738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:13.609683 3738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "e9a22350b1e44dcc935e18a905203e01"
format_stamp: "Formatted at 2026-08-06 00:54:13 on dist-test-slave-sb2z"
server_key: "5d365f5567f231c9cb0dd5285d050ea1"
server_key_iv: "6b41d42fac7f756ea48d9c9a264521a0"
server_key_version: "encryptionkey@0"
I20260806 00:54:13.610224 3738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "e9a22350b1e44dcc935e18a905203e01"
format_stamp: "Formatted at 2026-08-06 00:54:13 on dist-test-slave-sb2z"
server_key: "5d365f5567f231c9cb0dd5285d050ea1"
server_key_iv: "6b41d42fac7f756ea48d9c9a264521a0"
server_key_version: "encryptionkey@0"
I20260806 00:54:13.614251 3738 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260806 00:54:13.616956 3759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:13.618355 3738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 00:54:13.618505 3738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "e9a22350b1e44dcc935e18a905203e01"
format_stamp: "Formatted at 2026-08-06 00:54:13 on dist-test-slave-sb2z"
server_key: "5d365f5567f231c9cb0dd5285d050ea1"
server_key_iv: "6b41d42fac7f756ea48d9c9a264521a0"
server_key_version: "encryptionkey@0"
I20260806 00:54:13.618628 3738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:13.635147 3738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:13.638616 3738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:13.638851 3738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:13.646510 3738 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:38937
I20260806 00:54:13.646540 3829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:38937 every 8 connection(s)
I20260806 00:54:13.647676 3738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:13.647893 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 3738
I20260806 00:54:13.648093 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:54:13.648484 27959 external_mini_cluster.cc:1428] Setting key 771c757f4dd81be3e127ff02772f248b
I20260806 00:54:13.650898 3830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:13.658115 3830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01: Bootstrap starting.
Aug 06 00:54:13 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
I20260806 00:54:13.661163 3830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:13.662168 3830 log.cc:824] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:13.664366 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:13.650039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.27.77.254:38937)
0806 00:54:13.650476 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:13.650507 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:13.650546 (+ 39us) server_negotiation.cc:407] Connection header received
0806 00:54:13.651280 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:13.651300 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:13.651710 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:13.651997 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:13.652985 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:13.653996 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:13.655232 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:13.655828 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:13.655955 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:13.658960 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:13.658966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:13.658982 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:13.659015 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:13.661028 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:13.661533 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:13.661539 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:13.661546 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:13.661690 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:13.661963 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:13.661967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:13.661969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:13.662537 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:13.662722 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:13.663019 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 00:54:13.663360 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":149,"threads_started":1}
I20260806 00:54:13.664765 3830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01: No bootstrap required, opened a new log
I20260806 00:54:13.668715 3830 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } }
I20260806 00:54:13.668937 3830 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:13.668973 3830 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a22350b1e44dcc935e18a905203e01, State: Initialized, Role: FOLLOWER
I20260806 00:54:13.669507 3830 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } }
I20260806 00:54:13.669683 3830 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:13.669786 3830 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:13.669895 3830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:13.670867 3830 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } }
I20260806 00:54:13.671224 3830 leader_election.cc:302] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9a22350b1e44dcc935e18a905203e01; no voters:
I20260806 00:54:13.671607 3830 leader_election.cc:288] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:54:13.671764 3839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:13.672022 3839 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [term 1 LEADER]: Becoming Leader. State: Replica: e9a22350b1e44dcc935e18a905203e01, State: Running, Role: LEADER
I20260806 00:54:13.672374 3839 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } }
I20260806 00:54:13.672953 3830 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:13.674064 3841 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } }
I20260806 00:54:13.674191 3841 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:13.674605 3850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:13.674592 3842 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9a22350b1e44dcc935e18a905203e01. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } }
I20260806 00:54:13.674759 3842 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:13.678155 3850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:13.684419 3850 catalog_manager.cc:1347] Generated new cluster ID: 382f6c59a82144088da901f2ccc4a5ad
I20260806 00:54:13.684512 3850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:13.702610 3850 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:54:13.703701 3850 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:13.708034 3850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01: Generated new TSK 0
I20260806 00:54:13.708792 3850 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.193.keytab.
Entry for principal oryx/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.193.keytab.
I20260806 00:54:13.751227 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38937
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.193.keytab
--principal=oryx/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:13.880651 3864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:13.880959 3864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:13.881045 3864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:13.885233 3864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:13.885300 3864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:13.885392 3864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:54:13.891299 3864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.193.keytab
--principal=oryx/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:13.892459 3864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:13.893525 3864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:13.903625 3873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:13.903625 3876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:13.903654 3871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:13.904409 3864 server_base.cc:1034] running on GCE node
I20260806 00:54:13.904821 3864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:13.905476 3864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:13.906651 3864 hybrid_clock.cc:648] HybridClock initialized: now 1785977653906628 us; error 33 us; skew 500 ppm
Aug 06 00:54:13 dist-test-slave-sb2z krb5kdc[3718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977653, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:13.912262 3864 init.cc:377] Logged in from keytab as oryx/127.27.77.193@KRBTEST.COM (short username oryx)
I20260806 00:54:13.913259 3864 webserver.cc:460] Webserver started at http://127.27.77.193:33359/ using document root <none> and password file <none>
I20260806 00:54:13.913863 3864 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:13.913926 3864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:13.914104 3864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:13.916349 3864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda"
format_stamp: "Formatted at 2026-08-06 00:54:13 on dist-test-slave-sb2z"
server_key: "2a8547107e6f81b0f2e0100c73f508c2"
server_key_iv: "4a0c3a9abc41ae727b3d9877f2c1a611"
server_key_version: "encryptionkey@0"
I20260806 00:54:13.916885 3864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda"
format_stamp: "Formatted at 2026-08-06 00:54:13 on dist-test-slave-sb2z"
server_key: "2a8547107e6f81b0f2e0100c73f508c2"
server_key_iv: "4a0c3a9abc41ae727b3d9877f2c1a611"
server_key_version: "encryptionkey@0"
I20260806 00:54:13.920755 3864 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:54:13.923610 3885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:13.924945 3864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 00:54:13.925067 3864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda"
format_stamp: "Formatted at 2026-08-06 00:54:13 on dist-test-slave-sb2z"
server_key: "2a8547107e6f81b0f2e0100c73f508c2"
server_key_iv: "4a0c3a9abc41ae727b3d9877f2c1a611"
server_key_version: "encryptionkey@0"
I20260806 00:54:13.925164 3864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:13.986114 3864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:13.989554 3864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:13.989727 3864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:13.990593 3864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:13.991665 3864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:13.991721 3864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:13.991765 3864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:13.991780 3864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:14.001749 3864 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:39337
I20260806 00:54:14.001771 4040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:39337 every 8 connection(s)
I20260806 00:54:14.002700 3864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:54:14.011276 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 3864
I20260806 00:54:14.011421 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:54:14.011817 27959 external_mini_cluster.cc:1428] Setting key 00af6d3a5445ab9ad8ca3a2659df22e8
Aug 06 00:54:14 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977653, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
I20260806 00:54:14.018175 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.004665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:38741 (local address 127.27.77.254:38937)
0806 00:54:14.004818 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:14.004822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:14.005768 (+ 946us) server_negotiation.cc:407] Connection header received
0806 00:54:14.007297 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:14.007301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:14.007363 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:14.007444 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:14.008848 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.009437 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.010654 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.010896 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.010982 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.014195 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:14.014199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:14.014202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:14.014237 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:14.015956 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.016533 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.016537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.016539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.016631 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.017013 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.017016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.017018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.017211 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:14.017325 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:14.017847 (+ 522us) server_negotiation.cc:299] Negotiation successful
0806 00:54:14.017995 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:54:14.019111 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.005084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.193:38741)
0806 00:54:14.005605 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:14.005653 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:14.006993 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:14.007724 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:14.007731 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:14.008172 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:14.008680 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.008694 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.009570 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.009573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.010528 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.010539 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.011098 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.011103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.011329 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.012105 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:14.012122 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:14.014020 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:14.016094 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.016101 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.016118 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.016410 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.016743 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.016746 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.016748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.016904 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.017324 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:14.017332 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:14.017601 (+ 269us) client_negotiation.cc:769] Sending connection context
0806 00:54:14.017804 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 00:54:14.018152 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":129,"threads_started":1}
I20260806 00:54:14.020654 4041 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38937
I20260806 00:54:14.020955 4041 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:14.021618 4041 heartbeater.cc:511] Master 127.27.77.254:38937 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.27.77.194@KRBTEST.COM; defaulting to no policy
I20260806 00:54:14.023449 3780 ts_manager.cc:194] Registered new tserver with Master: 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda (127.27.77.193:39337)
I20260806 00:54:14.024926 3780 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:38741
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.77.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.194.keytab.
Entry for principal oryx/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.194.keytab.
I20260806 00:54:14.043738 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38937
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.194.keytab
--principal=oryx/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:14.163784 4048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:14.164069 4048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:14.164116 4048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:14.168210 4048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:14.168273 4048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:14.168334 4048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:54:14.173873 4048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.194.keytab
--principal=oryx/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:14.175122 4048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:14.176237 4048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:14.186443 4056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:14.186437 4054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:14.186437 4053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:14.186693 4048 server_base.cc:1034] running on GCE node
I20260806 00:54:14.187237 4048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:14.188058 4048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:14.189278 4048 hybrid_clock.cc:648] HybridClock initialized: now 1785977654189259 us; error 37 us; skew 500 ppm
Aug 06 00:54:14 dist-test-slave-sb2z krb5kdc[3718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977654, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:14.192385 4048 init.cc:377] Logged in from keytab as oryx/127.27.77.194@KRBTEST.COM (short username oryx)
I20260806 00:54:14.193331 4048 webserver.cc:460] Webserver started at http://127.27.77.194:37613/ using document root <none> and password file <none>
I20260806 00:54:14.193929 4048 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:14.193993 4048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:14.194249 4048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:14.196501 4048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "e3096c6aa5f340f58b7d3b0a68711297"
format_stamp: "Formatted at 2026-08-06 00:54:14 on dist-test-slave-sb2z"
server_key: "99f9452295105ab8121a3cd5296821e2"
server_key_iv: "9e5e59d615bb89a61ac554219bbbd37a"
server_key_version: "encryptionkey@0"
I20260806 00:54:14.197120 4048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "e3096c6aa5f340f58b7d3b0a68711297"
format_stamp: "Formatted at 2026-08-06 00:54:14 on dist-test-slave-sb2z"
server_key: "99f9452295105ab8121a3cd5296821e2"
server_key_iv: "9e5e59d615bb89a61ac554219bbbd37a"
server_key_version: "encryptionkey@0"
I20260806 00:54:14.201319 4048 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:54:14.203864 4067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:14.205206 4048 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:54:14.205355 4048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "e3096c6aa5f340f58b7d3b0a68711297"
format_stamp: "Formatted at 2026-08-06 00:54:14 on dist-test-slave-sb2z"
server_key: "99f9452295105ab8121a3cd5296821e2"
server_key_iv: "9e5e59d615bb89a61ac554219bbbd37a"
server_key_version: "encryptionkey@0"
I20260806 00:54:14.205473 4048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:14.233650 4048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:14.237268 4048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:14.237493 4048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:14.238164 4048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:14.239059 4048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:14.239109 4048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:14.239151 4048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:14.239207 4048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:14.248837 4048 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:44163
I20260806 00:54:14.248925 4212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:44163 every 8 connection(s)
I20260806 00:54:14.249809 4048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:54:14.251705 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 4048
I20260806 00:54:14.251835 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:54:14.252120 27959 external_mini_cluster.cc:1428] Setting key b3d36f08bf3a7092383016ff03420bc8
Aug 06 00:54:14 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977654, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.27.77.195@KRBTEST.COM; defaulting to no policy
I20260806 00:54:14.266217 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.251808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:51899 (local address 127.27.77.254:38937)
0806 00:54:14.251961 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:14.251966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:14.252990 (+ 1024us) server_negotiation.cc:407] Connection header received
0806 00:54:14.254524 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:14.254531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:14.254597 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:14.254693 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:14.256154 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.256768 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.258602 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.258830 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.258921 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.262054 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:14.262060 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:14.262064 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:14.262111 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:14.263851 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.264444 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.264447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.264448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.264533 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.264978 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.264981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.264983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.265210 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:14.265305 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:14.265844 (+ 539us) server_negotiation.cc:299] Negotiation successful
0806 00:54:14.266087 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.77.195@KRBTEST.COM" created.
I20260806 00:54:14.267221 4215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.252330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.194:51899)
0806 00:54:14.252829 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:14.252883 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:14.254303 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:14.254859 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:14.254866 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:14.255265 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:14.255954 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.255975 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.256920 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.256925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.258449 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.258466 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.259019 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.259024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.259295 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.260027 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:14.260055 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:14.261860 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:14.264037 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.264042 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.264061 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.264330 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.264645 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.264648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.264649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.264786 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.265329 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:14.265335 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:14.265584 (+ 249us) client_negotiation.cc:769] Sending connection context
0806 00:54:14.265759 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 00:54:14.266096 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":146,"threads_started":1}
I20260806 00:54:14.268596 4213 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38937
I20260806 00:54:14.268858 4213 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:14.269412 4213 heartbeater.cc:511] Master 127.27.77.254:38937 requested a full tablet report, sending...
I20260806 00:54:14.270488 3780 ts_manager.cc:194] Registered new tserver with Master: e3096c6aa5f340f58b7d3b0a68711297 (127.27.77.194:44163)
I20260806 00:54:14.271085 3780 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:51899
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.195.keytab.
Entry for principal oryx/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.195.keytab.
I20260806 00:54:14.286255 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38937
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.195.keytab
--principal=oryx/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:14.402403 4222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:14.402717 4222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:14.402837 4222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:14.406873 4222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:14.406962 4222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:14.407050 4222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:54:14.412529 4222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.195.keytab
--principal=oryx/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:14.413698 4222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:14.414776 4222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:14.425074 4235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:14.425154 4231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:14.425068 4229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:14.425470 4222 server_base.cc:1034] running on GCE node
I20260806 00:54:14.425868 4222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:14.426565 4222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:14.427763 4222 hybrid_clock.cc:648] HybridClock initialized: now 1785977654427754 us; error 31 us; skew 500 ppm
Aug 06 00:54:14 dist-test-slave-sb2z krb5kdc[3718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977654, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:14.430861 4222 init.cc:377] Logged in from keytab as oryx/127.27.77.195@KRBTEST.COM (short username oryx)
I20260806 00:54:14.431747 4222 webserver.cc:460] Webserver started at http://127.27.77.195:39367/ using document root <none> and password file <none>
I20260806 00:54:14.432276 4222 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:14.432333 4222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:14.432499 4222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:14.434489 4222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "cb3523d7e6f84f349d3bb5c864757290"
format_stamp: "Formatted at 2026-08-06 00:54:14 on dist-test-slave-sb2z"
server_key: "4e09793bf0ba21f9f308d86dacae935c"
server_key_iv: "4716c018ac852736dcf9fbcafadb69d4"
server_key_version: "encryptionkey@0"
I20260806 00:54:14.435005 4222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "cb3523d7e6f84f349d3bb5c864757290"
format_stamp: "Formatted at 2026-08-06 00:54:14 on dist-test-slave-sb2z"
server_key: "4e09793bf0ba21f9f308d86dacae935c"
server_key_iv: "4716c018ac852736dcf9fbcafadb69d4"
server_key_version: "encryptionkey@0"
I20260806 00:54:14.438799 4222 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:54:14.441254 4245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:14.442584 4222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:54:14.442724 4222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "cb3523d7e6f84f349d3bb5c864757290"
format_stamp: "Formatted at 2026-08-06 00:54:14 on dist-test-slave-sb2z"
server_key: "4e09793bf0ba21f9f308d86dacae935c"
server_key_iv: "4716c018ac852736dcf9fbcafadb69d4"
server_key_version: "encryptionkey@0"
I20260806 00:54:14.442837 4222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:14.471972 4222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:14.475833 4222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:14.476063 4222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:14.476724 4222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:14.477607 4222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:14.477665 4222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:14.477738 4222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:14.477773 4222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:14.487003 4222 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:42489
I20260806 00:54:14.487035 4405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:42489 every 8 connection(s)
I20260806 00:54:14.487922 4222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:54:14.494062 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 4222
I20260806 00:54:14.494174 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:54:14.494485 27959 external_mini_cluster.cc:1428] Setting key 64235311da900bd3d922f2478684b976
Aug 06 00:54:14 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977654, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
I20260806 00:54:14.502924 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.489872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:42319 (local address 127.27.77.254:38937)
0806 00:54:14.490011 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:14.490015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:14.490912 (+ 897us) server_negotiation.cc:407] Connection header received
0806 00:54:14.492431 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:14.492434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:14.492494 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:14.492608 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:14.493778 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.494433 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.495995 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.496303 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.496394 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.499467 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:14.499471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:14.499474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:14.499528 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:14.500941 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.501451 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.501454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.501455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.501531 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.501865 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.501867 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.501868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.502035 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:14.502113 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:14.502691 (+ 578us) server_negotiation.cc:299] Negotiation successful
0806 00:54:14.502804 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:54:14.503945 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.490302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.195:42319)
0806 00:54:14.490764 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:14.490806 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:14.492218 (+ 1412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:14.492784 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:14.492790 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:14.493158 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:14.493629 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.493643 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.494569 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.494573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.495824 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.495837 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.496461 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.496465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.496691 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.497326 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:14.497342 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:14.499268 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:14.501051 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.501055 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.501067 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.501359 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.501635 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.501638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.501639 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.501781 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.502132 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:14.502138 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:14.502411 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 00:54:14.502600 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 00:54:14.502888 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":132,"threads_started":1}
I20260806 00:54:14.505174 4406 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38937
I20260806 00:54:14.505432 4406 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:14.505949 4406 heartbeater.cc:511] Master 127.27.77.254:38937 requested a full tablet report, sending...
I20260806 00:54:14.507074 3780 ts_manager.cc:194] Registered new tserver with Master: cb3523d7e6f84f349d3bb5c864757290 (127.27.77.195:42489)
I20260806 00:54:14.507784 3780 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:42319
I20260806 00:54:14.508817 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.27.77.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.77.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.77.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.253.keytab.
Entry for principal oryx/127.27.77.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.253.keytab.
I20260806 00:54:14.538383 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.253:45881
--webserver_interface=127.27.77.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.253.keytab
--principal=oryx/127.27.77.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:38937,127.27.77.253:45881
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:14.652710 4412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:14.652953 4412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:14.652995 4412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:14.657131 4412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:14.657270 4412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:14.657323 4412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:14.657366 4412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:14.657403 4412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:54:14.662657 4412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:38937,127.27.77.253:45881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.253.keytab
--principal=oryx/127.27.77.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.253:45881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.77.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
W20260806 00:54:14.662925 4412 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.27.77.254:38937,127.27.77.253:45881'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 00:54:14.663825 4412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:14.664902 4412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:14.673411 4424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:14.673672 4419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:14.674047 4412 server_base.cc:1034] running on GCE node
W20260806 00:54:14.674163 4420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:14.674789 4412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:14.676081 4412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:14.677255 4412 hybrid_clock.cc:648] HybridClock initialized: now 1785977654677226 us; error 40 us; skew 500 ppm
Aug 06 00:54:14 dist-test-slave-sb2z krb5kdc[3718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977654, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:14.680155 4412 init.cc:377] Logged in from keytab as oryx/127.27.77.253@KRBTEST.COM (short username oryx)
I20260806 00:54:14.681066 4412 webserver.cc:460] Webserver started at http://127.27.77.253:37479/ using document root <none> and password file <none>
I20260806 00:54:14.681591 4412 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:14.681648 4412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:14.681813 4412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:14.683887 4412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/data/instance:
uuid: "bbdf6367d78047ddafb01c62630ff258"
format_stamp: "Formatted at 2026-08-06 00:54:14 on dist-test-slave-sb2z"
server_key: "504eb2de3046c1006683942f04f1e7d9"
server_key_iv: "720827cb4a251cf7cf7188099ea28dac"
server_key_version: "encryptionkey@0"
I20260806 00:54:14.684432 4412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/wal/instance:
uuid: "bbdf6367d78047ddafb01c62630ff258"
format_stamp: "Formatted at 2026-08-06 00:54:14 on dist-test-slave-sb2z"
server_key: "504eb2de3046c1006683942f04f1e7d9"
server_key_iv: "720827cb4a251cf7cf7188099ea28dac"
server_key_version: "encryptionkey@0"
I20260806 00:54:14.688237 4412 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:54:14.690760 4434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:14.692220 4412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:54:14.692374 4412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/wal
uuid: "bbdf6367d78047ddafb01c62630ff258"
format_stamp: "Formatted at 2026-08-06 00:54:14 on dist-test-slave-sb2z"
server_key: "504eb2de3046c1006683942f04f1e7d9"
server_key_iv: "720827cb4a251cf7cf7188099ea28dac"
server_key_version: "encryptionkey@0"
I20260806 00:54:14.692492 4412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:14.711779 4412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:14.715261 4412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:14.715481 4412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:14.722983 4412 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.253:45881
I20260806 00:54:14.723287 4505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.253:45881 every 8 connection(s)
I20260806 00:54:14.724171 4412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/data/info.pb
I20260806 00:54:14.725376 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 4412
I20260806 00:54:14.727473 4506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:14.731441 4506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:54:14 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.77.253@KRBTEST.COM
Aug 06 00:54:14 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977654, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.253@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
I20260806 00:54:14.741818 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.726972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53508 (local address 127.27.77.253:45881)
0806 00:54:14.727416 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:14.727431 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:14.727466 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:54:14.728197 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:14.728213 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:14.728533 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:14.728931 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:14.729788 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.730816 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.731773 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.732298 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.732481 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.735589 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:14.735604 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:14.735616 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:14.735650 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:14.738530 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.739087 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.739093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.739100 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.739247 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.739505 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.739509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.739510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.740033 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:14.740224 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:14.740569 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 00:54:14.740883 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":143,"threads_started":1}
I20260806 00:54:14.743345 4513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.733191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.0.0.1:42890)
0806 00:54:14.733610 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:14.733674 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:14.733814 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:14.734135 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:14.734142 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:14.734561 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:14.734740 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.734746 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.735587 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.735592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.736590 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.736602 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.737241 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.737247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.737384 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.737916 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:14.737931 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:14.739775 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:14.741599 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.741607 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.741612 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.741900 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.742257 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.742259 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.742261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.742348 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.742784 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:14.742790 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:14.742914 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 00:54:14.743057 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 00:54:14.743210 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":142,"threads_started":1}
I20260806 00:54:14.743427 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.733259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.27.77.254:38937)
0806 00:54:14.733421 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:14.733426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:14.733769 (+ 343us) server_negotiation.cc:407] Connection header received
0806 00:54:14.733928 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:14.733930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:14.733975 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:14.734048 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:14.734873 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.735424 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.736716 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.737076 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.737186 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.739926 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:14.739928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:14.739930 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:14.739956 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:14.741490 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.742028 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.742032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.742034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.742126 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.742443 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.742446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.742447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.742671 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:14.742754 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:14.743158 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 00:54:14.743279 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 00:54:14.744568 4506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:54:14 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977654, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.253@KRBTEST.COM for oryx/127.27.77.253@KRBTEST.COM
I20260806 00:54:14.753000 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.745240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53512 (local address 127.27.77.253:45881)
0806 00:54:14.745347 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:14.745351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:14.745364 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:54:14.745442 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:14.745447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:14.745487 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:14.745577 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:14.745952 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.746861 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.747802 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.748017 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.748121 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.750006 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:14.750010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:14.750012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:14.750034 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:14.751376 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.751824 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.751827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.751828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.751904 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.752182 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.752185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.752186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.752347 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:14.752422 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:14.752716 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 00:54:14.752808 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 00:54:14.753014 4513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.745131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:45881 (local address 127.0.0.1:53512)
0806 00:54:14.745245 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:14.745281 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:14.745364 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:14.745588 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:14.745590 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:14.745753 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:14.745838 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.745843 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.746992 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.746995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.747678 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.747688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.748132 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.748136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.748225 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.748618 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:14.748629 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:14.749880 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:14.751482 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.751484 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.751485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.751739 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.751987 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.751989 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.751990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.752100 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.752456 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:14.752459 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:14.752557 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 00:54:14.752657 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 00:54:14.752763 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 00:54:14.756135 4506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: Bootstrap starting.
I20260806 00:54:14.758524 4506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:14.759267 4506 log.cc:824] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:14.761297 4506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: No bootstrap required, opened a new log
I20260806 00:54:14.764348 4506 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } }
I20260806 00:54:14.764573 4506 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:14.764658 4506 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbdf6367d78047ddafb01c62630ff258, State: Initialized, Role: FOLLOWER
I20260806 00:54:14.765092 4506 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } }
I20260806 00:54:14.765579 4515 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } } }
I20260806 00:54:14.765702 4515 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:54:14.766286 4506 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 00:54:14.771431 4530 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:54:14.771559 4530 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:54:14.773861 4412 master_runner.cc:428] Detected that this master bbdf6367d78047ddafb01c62630ff258 is joining an existing cluster
I20260806 00:54:14.773936 4412 master_runner.cc:432] Initiating AddMaster RPC to add 127.27.77.253:45881
I20260806 00:54:14.784947 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.776387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53526 (local address 127.27.77.253:45881)
0806 00:54:14.776772 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:14.776776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:14.776998 (+ 222us) server_negotiation.cc:407] Connection header received
0806 00:54:14.777127 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:14.777129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:14.777173 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:14.777244 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:14.777849 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.778565 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.779466 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.779785 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.779883 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.781142 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:14.781147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:14.781153 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:14.781180 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:14.783200 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.783755 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.783759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.783760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.783837 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.784099 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.784102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.784105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.784309 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:14.784371 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:14.784654 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 00:54:14.784778 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311}
I20260806 00:54:14.785147 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.776454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42902 (local address 127.27.77.254:38937)
0806 00:54:14.776584 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:14.776589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:14.776774 (+ 185us) server_negotiation.cc:407] Connection header received
0806 00:54:14.777025 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:14.777029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:14.777085 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:14.777174 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:14.777690 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.778536 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.779780 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.780070 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.780183 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.781691 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:14.781694 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:14.781697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:14.781722 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:14.783316 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.783853 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.783857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.783859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.783956 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.784269 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.784273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.784274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.784490 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:14.784601 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:14.784859 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 00:54:14.784988 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 00:54:14.785486 4536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.776379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.0.0.1:42902)
0806 00:54:14.776687 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:14.776703 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:14.776848 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:14.777199 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:14.777202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:14.777435 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:14.777562 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.777568 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.778663 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.778667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.779660 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.779671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.780189 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.780483 (+ 294us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.780560 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.781021 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:14.781043 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:14.781556 (+ 513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:14.783443 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.783446 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.783447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.783754 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.784067 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.784073 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.784078 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.784166 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.784604 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:14.784609 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:14.784682 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 00:54:14.784784 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 00:54:14.784903 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":28,"thread_start_us":119,"threads_started":1}
I20260806 00:54:14.785771 4537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.776626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:45881 (local address 127.0.0.1:53526)
0806 00:54:14.776912 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:14.776929 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:14.777038 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:14.777342 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:14.777345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:14.777608 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:14.777730 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.777738 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.778696 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.778698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.779343 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.779351 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.779907 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.779910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.780013 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.780522 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:14.780538 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:14.780999 (+ 461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:14.783318 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.783321 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.783323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.783653 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.783931 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.783933 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.783934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.783994 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.784413 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:14.784415 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:14.784492 (+ 77us) client_negotiation.cc:769] Sending connection context
0806 00:54:14.785292 (+ 800us) client_negotiation.cc:240] Negotiation successful
0806 00:54:14.785419 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":41,"thread_start_us":94,"threads_started":1}
Aug 06 00:54:14 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977653, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.254@KRBTEST.COM for oryx/127.27.77.253@KRBTEST.COM
I20260806 00:54:14.800510 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.790084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53534 (local address 127.27.77.253:45881)
0806 00:54:14.790227 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:14.790232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:14.790626 (+ 394us) server_negotiation.cc:407] Connection header received
0806 00:54:14.790810 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:14.790812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:14.790859 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:14.790926 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:14.791759 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.792236 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.793489 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.793694 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.793777 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.796511 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:14.796514 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:14.796516 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:14.796538 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:14.798412 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.799061 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.799066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.799068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.799162 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.799487 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.799492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.799494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.799742 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:14.799858 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:14.800226 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 00:54:14.800348 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:54:14.800526 4538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.790053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:45881 (local address 127.0.0.1:53534)
0806 00:54:14.790468 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:14.790528 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:14.790671 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:14.790999 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:14.791005 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:14.791449 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:14.791636 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.791644 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.792345 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.792348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.793360 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.793372 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.793810 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.793813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.793938 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.794474 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:14.794489 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:14.796382 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:14.798555 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.798568 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.798575 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.798923 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.799289 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.799294 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.799296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.799385 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.799871 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:14.799880 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:14.800003 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 00:54:14.800158 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 00:54:14.800362 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":124,"threads_started":1}
I20260806 00:54:14.802328 3780 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.27.77.253:45881: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bbdf6367d78047ddafb01c62630ff258"
member_type: NON_VOTER
last_known_addr {
host: "127.27.77.253"
port: 45881
}
attrs {
promote: true
}
}
dest_uuid: "e9a22350b1e44dcc935e18a905203e01"
cas_config_opid_index: -1
I20260806 00:54:14.802997 3780 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } }
I20260806 00:54:14.804879 4542 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bbdf6367d78047ddafb01c62630ff258 (127.27.77.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } } }
I20260806 00:54:14.805817 4468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 0 FOLLOWER]: Advancing to term 1
WARNING: no policy specified for oryx/127.27.77.252@KRBTEST.COM; defaulting to no policy
I20260806 00:54:14.806878 4542 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.27.77.253:45881
I20260806 00:54:14.807359 4543 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } } }
I20260806 00:54:14.807475 4543 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:14.807636 4412 master.cc:537] Master@127.27.77.253:45881 shutting down...
I20260806 00:54:14.808466 4468 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 FOLLOWER]: Refusing update from remote peer e9a22350b1e44dcc935e18a905203e01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 00:54:14.809216 4543 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.77.252@KRBTEST.COM" created.
W20260806 00:54:14.811260 3766 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 00:54:14.812736 3766 consensus_peers.cc:487] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 -> Peer bbdf6367d78047ddafb01c62630ff258 (127.27.77.253:45881): Couldn't send request to peer bbdf6367d78047ddafb01c62630ff258. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 00:54:14.812839 4412 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:14.813127 4412 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:14.813233 4412 tablet_replica.cc:330] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: stopping tablet replica
I20260806 00:54:14.829003 4412 master.cc:552] Master@127.27.77.253:45881 shutdown complete.
I20260806 00:54:14.829089 4412 master_runner.cc:305] Clearing existing system tablet
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.77.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.252.keytab.
Entry for principal oryx/127.27.77.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.252.keytab.
I20260806 00:54:14.829361 4412 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:14.829435 4412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:14.830423 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.252:43643
--webserver_interface=127.27.77.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.252.keytab
--principal=oryx/127.27.77.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:38937,127.27.77.253:45881,127.27.77.252:43643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:54:14.831136 4412 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 00:54:14.832144 4550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:14.832435 4412 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:14.832593 4412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 00:54:14.832634 4412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/wal
uuid: "bbdf6367d78047ddafb01c62630ff258"
format_stamp: "Formatted at 2026-08-06 00:54:14 on dist-test-slave-sb2z"
server_key: "504eb2de3046c1006683942f04f1e7d9"
server_key_iv: "720827cb4a251cf7cf7188099ea28dac"
server_key_version: "encryptionkey@0"
I20260806 00:54:14.834385 4412 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:14.836498 4412 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 00:54:14.836618 4412 log.cc:1196] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 00:54:14.836869 4412 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: Deleting consensus metadata
I20260806 00:54:14.837066 4412 master_runner.cc:315] Copying system tablet from 127.27.77.254:38937
I20260806 00:54:14.838248 4412 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: tablet copy: Beginning tablet copy session from remote peer at address 127.27.77.254:38937
I20260806 00:54:14.846822 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.838945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42906 (local address 127.27.77.254:38937)
0806 00:54:14.839091 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:14.839108 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:14.839307 (+ 199us) server_negotiation.cc:407] Connection header received
0806 00:54:14.839471 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:14.839474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:14.839572 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:14.839655 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:14.840161 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.840695 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.841847 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.842080 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.842195 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.843366 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:14.843369 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:14.843372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:14.843399 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:14.844973 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.845426 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.845429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.845431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.845520 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:14.845850 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:14.845854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:14.845856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:14.846042 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:14.846159 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:14.846527 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 00:54:14.846650 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:54:14.846832 4558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:14.838875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.0.0.1:42906)
0806 00:54:14.839203 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:14.839215 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:14.839360 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:14.839677 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:14.839680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:14.839909 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:14.840019 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.840025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.840813 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.840817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.841702 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:14.841712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:14.842215 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:14.842219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:14.842321 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:14.842875 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:14.842887 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:14.843217 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:14.845094 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.845098 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.845101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.845318 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.845657 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:14.845660 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:14.845662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:14.845752 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:14.846167 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:14.846173 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:14.846278 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 00:54:14.846468 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 00:54:14.846630 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":86,"threads_started":1}
I20260806 00:54:14.847815 3806 tablet_copy_service.cc:140] P e9a22350b1e44dcc935e18a905203e01: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bbdf6367d78047ddafb01c62630ff258 ({username='oryx', principal='oryx/127.27.77.253@KRBTEST.COM'} at 127.0.0.1:42906)
I20260806 00:54:14.848017 3806 tablet_copy_service.cc:161] P e9a22350b1e44dcc935e18a905203e01: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bbdf6367d78047ddafb01c62630ff258 at {username='oryx', principal='oryx/127.27.77.253@KRBTEST.COM'} at 127.0.0.1:42906: session id = bbdf6367d78047ddafb01c62630ff258-00000000000000000000000000000000
I20260806 00:54:14.849905 3806 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 00:54:14.851680 4412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 00:54:14.853643 4412 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: tablet copy: Starting download of 0 data blocks...
I20260806 00:54:14.853865 4412 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: tablet copy: Starting download of 1 WAL segments...
I20260806 00:54:14.855647 4412 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 00:54:14.857318 3806 tablet_copy_service.cc:342] P e9a22350b1e44dcc935e18a905203e01: Request end of tablet copy session bbdf6367d78047ddafb01c62630ff258-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.27.77.253@KRBTEST.COM'} at 127.0.0.1:42906
I20260806 00:54:14.857445 3806 tablet_copy_service.cc:433] P e9a22350b1e44dcc935e18a905203e01: ending tablet copy session bbdf6367d78047ddafb01c62630ff258-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bbdf6367d78047ddafb01c62630ff258
I20260806 00:54:14.864611 4412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:14.866288 4559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:14.866395 4562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:14.866626 4412 server_base.cc:1034] running on GCE node
W20260806 00:54:14.866873 4560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:14.867940 4412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:14.868289 4412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:14.869446 4412 hybrid_clock.cc:648] HybridClock initialized: now 1785977654869455 us; error 45 us; skew 500 ppm
Aug 06 00:54:14 dist-test-slave-sb2z krb5kdc[3718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977654, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:14.871006 4412 init.cc:377] Logged in from keytab as oryx/127.27.77.253@KRBTEST.COM (short username oryx)
I20260806 00:54:14.871502 4412 webserver.cc:460] Webserver started at http://127.27.77.253:35539/ using document root <none> and password file <none>
I20260806 00:54:14.871723 4412 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:14.871783 4412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:14.872939 4412 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 00:54:14.874045 4570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:14.874359 4412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 00:54:14.874439 4412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/wal
uuid: "bbdf6367d78047ddafb01c62630ff258"
format_stamp: "Formatted at 2026-08-06 00:54:14 on dist-test-slave-sb2z"
server_key: "504eb2de3046c1006683942f04f1e7d9"
server_key_iv: "720827cb4a251cf7cf7188099ea28dac"
server_key_version: "encryptionkey@0"
I20260806 00:54:14.874540 4412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:14.893031 4412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:14.895169 4412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:14.900753 4412 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.253:45881
I20260806 00:54:14.901077 4637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.253:45881 every 8 connection(s)
I20260806 00:54:14.901446 4412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-1/data/info.pb
I20260806 00:54:14.902997 4639 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:54:14.903913 4639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: Bootstrap starting.
I20260806 00:54:14.912657 4639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:54:14.912971 4639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: Bootstrap complete.
I20260806 00:54:14.913553 4639 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } }
I20260806 00:54:14.913697 4639 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bbdf6367d78047ddafb01c62630ff258, State: Initialized, Role: LEARNER
I20260806 00:54:14.913817 4639 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } }
I20260806 00:54:14.914188 4642 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } } }
I20260806 00:54:14.914363 4642 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: This master's current role is: LEARNER
I20260806 00:54:14.914466 4639 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:14.917724 4654 catalog_manager.cc:1259] Loaded cluster ID: 382f6c59a82144088da901f2ccc4a5ad
I20260806 00:54:14.917835 4654 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: loading cluster ID for follower catalog manager: success
I20260806 00:54:14.919490 4654 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: acquiring CA information for follower catalog manager: success
I20260806 00:54:14.920614 4654 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 00:54:14.995095 4551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:14.995378 4551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:14.995420 4551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:14.999410 4551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:14.999476 4551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:14.999497 4551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:14.999513 4551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:14.999601 4551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:54:15.005831 4551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:38937,127.27.77.253:45881,127.27.77.252:43643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.252.keytab
--principal=oryx/127.27.77.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.252:43643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.77.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:15.006830 4551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:15.007871 4551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:54:15.017386 4551 server_base.cc:1034] running on GCE node
W20260806 00:54:15.017326 4663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:15.017328 4665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:15.017338 4661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:15.018184 4551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:15.019340 4551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:15.020519 4551 hybrid_clock.cc:648] HybridClock initialized: now 1785977655020500 us; error 34 us; skew 500 ppm
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:15.023617 4551 init.cc:377] Logged in from keytab as oryx/127.27.77.252@KRBTEST.COM (short username oryx)
I20260806 00:54:15.024559 4551 webserver.cc:460] Webserver started at http://127.27.77.252:45483/ using document root <none> and password file <none>
I20260806 00:54:15.025125 4551 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:15.025192 4551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:15.025447 4551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:15.027501 4551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/data/instance:
uuid: "e239ce00299a4509830746a4126c44cf"
format_stamp: "Formatted at 2026-08-06 00:54:15 on dist-test-slave-sb2z"
server_key: "658c939cc1d2cb95f777b30e76e685ea"
server_key_iv: "034242b0aba43a5cbc30133707205615"
server_key_version: "encryptionkey@0"
I20260806 00:54:15.027877 4041 heartbeater.cc:503] Master 127.27.77.254:38937 was elected leader, sending a full tablet report...
I20260806 00:54:15.028195 4551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/wal/instance:
uuid: "e239ce00299a4509830746a4126c44cf"
format_stamp: "Formatted at 2026-08-06 00:54:15 on dist-test-slave-sb2z"
server_key: "658c939cc1d2cb95f777b30e76e685ea"
server_key_iv: "034242b0aba43a5cbc30133707205615"
server_key_version: "encryptionkey@0"
I20260806 00:54:15.032179 4551 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:54:15.034576 4675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:15.035905 4551 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:54:15.036058 4551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/wal
uuid: "e239ce00299a4509830746a4126c44cf"
format_stamp: "Formatted at 2026-08-06 00:54:15 on dist-test-slave-sb2z"
server_key: "658c939cc1d2cb95f777b30e76e685ea"
server_key_iv: "034242b0aba43a5cbc30133707205615"
server_key_version: "encryptionkey@0"
I20260806 00:54:15.036178 4551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:15.057344 4551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:15.061012 4551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:15.061237 4551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:15.068871 4551 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.252:43643
I20260806 00:54:15.069190 4750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.252:43643 every 8 connection(s)
I20260806 00:54:15.069936 4551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/data/info.pb
I20260806 00:54:15.072890 4751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:15.077111 4751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:54:15.079376 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 4551
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.252@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.77.252@KRBTEST.COM
I20260806 00:54:15.094430 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.079161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42914 (local address 127.27.77.254:38937)
0806 00:54:15.079349 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.079354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.080190 (+ 836us) server_negotiation.cc:407] Connection header received
0806 00:54:15.081284 (+ 1094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.081288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.081356 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.081449 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.082263 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.082794 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.083899 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.084130 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.084245 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.090171 (+ 5926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.090174 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.090176 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.090203 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.091840 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.092586 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.092592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.092597 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.092695 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.093097 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.093102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.093107 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.093374 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.093480 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.094094 (+ 614us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.094250 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 00:54:15.095510 4755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.080476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44312 (local address 127.27.77.252:43643)
0806 00:54:15.080833 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.080843 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.080875 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:54:15.081101 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.081117 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.081310 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.081504 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.082367 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.083222 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.084108 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.084458 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.084584 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.090531 (+ 5947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.090537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.090549 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.090570 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.092422 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.092927 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.092933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.092935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.093021 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.093326 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.093330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.093332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.093804 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.093952 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.094249 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.094546 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":185,"server-negotiator.queue_time_us":263,"thread_start_us":149,"threads_started":1}
I20260806 00:54:15.095510 4753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.079337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.0.0.1:42914)
0806 00:54:15.079993 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.080038 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.081069 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.081504 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.081510 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.081867 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.082116 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.082129 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.082951 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.082954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.083769 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.083779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.084289 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.084293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.084457 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.088221 (+ 3764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.088246 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.089915 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.091967 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.091977 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.091989 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.092447 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.092797 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.092800 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.092802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.092975 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.093488 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.093494 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.093814 (+ 320us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.094013 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.094427 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":21,"spinlock_wait_cycles":4864,"thread_start_us":213,"threads_started":1}
I20260806 00:54:15.097043 4751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.252@KRBTEST.COM for oryx/127.27.77.253@KRBTEST.COM
I20260806 00:54:15.105186 4753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.097581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:45881 (local address 127.0.0.1:53540)
0806 00:54:15.097725 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.097741 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.097811 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.098205 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.098207 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.098341 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.098433 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.098438 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.099321 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.099323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.099961 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.099971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.100411 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.100413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.100487 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.100973 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.100985 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.102308 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.103961 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.103963 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.103965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.104168 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.104409 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.104411 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.104412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.104486 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.104803 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.104805 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.104883 (+ 78us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.104963 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.105064 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 00:54:15.105259 4756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.097716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53540 (local address 127.27.77.253:45881)
0806 00:54:15.097998 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.098002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.098014 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:54:15.098050 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.098052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.098089 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.098173 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.098534 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.099229 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.100072 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.100319 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.100396 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.102439 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.102442 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.102444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.102467 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.103864 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.104254 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.104256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.104257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.104330 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.104560 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.104562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.104563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.104724 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.104796 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.105016 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.105112 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":128,"threads_started":1}
I20260806 00:54:15.106025 4751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.252@KRBTEST.COM for oryx/127.27.77.252@KRBTEST.COM
I20260806 00:54:15.113720 4753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.106530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:43643 (local address 127.0.0.1:44318)
0806 00:54:15.106679 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.106691 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.106753 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.106991 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.106993 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.107136 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.107207 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.107212 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.108015 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.108018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.108502 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.108510 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.109018 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.109023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.109096 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.109591 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.109603 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.110822 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.112386 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.112389 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.112390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.112580 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.112874 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.112876 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.112877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.112962 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.113311 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.113313 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.113389 (+ 76us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.113485 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.113597 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 00:54:15.113806 4755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.106598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44318 (local address 127.27.77.252:43643)
0806 00:54:15.106721 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.106724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.106736 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:54:15.106827 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.106829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.106862 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.106925 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.107307 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.107899 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.108613 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.108860 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.108941 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.110947 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.110950 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.110952 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.110972 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.112281 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.112691 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.112694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.112695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.112764 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.113051 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.113054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.113055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.113221 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.113298 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.113529 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.113672 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":54}
I20260806 00:54:15.117743 4751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: Bootstrap starting.
I20260806 00:54:15.120342 4751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:15.121191 4751 log.cc:824] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:15.123437 4751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: No bootstrap required, opened a new log
I20260806 00:54:15.126819 4751 raft_consensus.cc:348] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } }
I20260806 00:54:15.127079 4751 raft_consensus.cc:374] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:15.127183 4751 raft_consensus.cc:729] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e239ce00299a4509830746a4126c44cf, State: Initialized, Role: FOLLOWER
I20260806 00:54:15.127731 4751 consensus_queue.cc:260] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } }
I20260806 00:54:15.128245 4759 sys_catalog.cc:455] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } } }
I20260806 00:54:15.128367 4759 sys_catalog.cc:458] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:54:15.129199 4751 sys_catalog.cc:564] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: configured and running, proceeding with master startup.
W20260806 00:54:15.134526 4773 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:54:15.134627 4773 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:54:15.138461 4551 master_runner.cc:428] Detected that this master e239ce00299a4509830746a4126c44cf is joining an existing cluster
I20260806 00:54:15.138525 4551 master_runner.cc:432] Initiating AddMaster RPC to add 127.27.77.252:43643
I20260806 00:54:15.149789 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 3864
I20260806 00:54:15.161634 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:39337
--local_ip_for_outbound_sockets=127.27.77.193
--tserver_master_addrs=127.27.77.254:38937
--webserver_port=33359
--webserver_interface=127.27.77.193
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.193.keytab
--principal=oryx/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.77.254:38937,127.27.77.253:45881
--tserver_master_addrs=127.27.77.254:38937,127.27.77.253:45881,127.27.77.252:43643 with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:54:15.170043 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.141407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42924 (local address 127.27.77.254:38937)
0806 00:54:15.141633 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.141639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.141661 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:54:15.141815 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.141820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.141875 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.141952 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.142780 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.143502 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.144836 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.145188 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.145263 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.146779 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.146782 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.146785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.146823 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.148585 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.168239 (+ 19654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.168245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.168246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.168387 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.168812 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.168817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.168821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.169091 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.169178 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.169665 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.169823 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:54:15.170755 4783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.140856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.0.0.1:42924)
0806 00:54:15.141599 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.141610 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.141690 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.142097 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.142100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.142415 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.142636 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.142644 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.143677 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.143680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.144650 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.144659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.145429 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.145433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.145605 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.146208 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.146227 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.146635 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.148706 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.148709 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.148711 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.168031 (+ 19320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.168584 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.168587 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.168589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.168687 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.169274 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.169279 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.169434 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.170315 (+ 881us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.170514 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":32,"spinlock_wait_cycles":4352,"thread_start_us":178,"threads_started":1}
I20260806 00:54:15.172575 4755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.141321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44332 (local address 127.27.77.252:43643)
0806 00:54:15.141431 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.141435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.141902 (+ 467us) server_negotiation.cc:407] Connection header received
0806 00:54:15.142103 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.142106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.142142 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.142187 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.143167 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.143963 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.145724 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.146084 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.146198 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.148406 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.148412 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.148417 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.148447 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.150327 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.171235 (+ 20908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.171239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.171241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.171315 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.171672 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.171675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.171677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.171884 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.171958 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.172271 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.172403 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38,"spinlock_wait_cycles":768}
I20260806 00:54:15.172957 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.141289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:45881 (local address 127.0.0.1:53542)
0806 00:54:15.141962 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.141975 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.142110 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.142491 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.142493 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.142732 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.143153 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.143161 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.144210 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.144213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.145079 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.145089 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.145814 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.145818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.145895 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.168248 (+ 22353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.168267 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.168758 (+ 491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.171171 (+ 2413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.171177 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.171182 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.171428 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.171774 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.171777 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.171779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.171879 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.172403 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.172407 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.172491 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.172593 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.172745 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":271,"spinlock_wait_cycles":1920,"thread_start_us":95,"threads_started":1}
I20260806 00:54:15.173326 4781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.140964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:43643 (local address 127.0.0.1:44332)
0806 00:54:15.141819 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.141833 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.142000 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.142306 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.142308 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.142602 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.143042 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.143055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.144094 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.144097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.145584 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.145596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.147158 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.147161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.147257 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.147774 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.147789 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.148256 (+ 467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.150455 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.150461 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.150466 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.171123 (+ 20657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.171440 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.171446 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.171451 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.171576 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.172018 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.172021 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.172110 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.172765 (+ 655us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.172928 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":31,"thread_start_us":111,"threads_started":1}
I20260806 00:54:15.173704 4756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.141028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53542 (local address 127.27.77.253:45881)
0806 00:54:15.141173 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.141178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.142039 (+ 861us) server_negotiation.cc:407] Connection header received
0806 00:54:15.142222 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.142228 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.142281 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.142335 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.143292 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.144069 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.145220 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.145560 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.145647 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.168976 (+ 23329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.168980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.168983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.169016 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.170809 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.171573 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.171577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.171579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.171668 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.172010 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.172013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.172015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.172247 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.172355 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.173379 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.173558 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977653, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.254@KRBTEST.COM for oryx/127.27.77.252@KRBTEST.COM
I20260806 00:54:15.185292 4538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.175693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:43643 (local address 127.0.0.1:44336)
0806 00:54:15.175863 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.175877 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.175958 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.176290 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.176293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.176492 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.176608 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.176615 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.177750 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.177755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.178679 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.178693 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.179343 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.179347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.179460 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.180048 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.180063 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.181640 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.183645 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.183649 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.183651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.183905 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.184226 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.184230 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.184232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.184332 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.184758 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.184761 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.184860 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.184980 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.185122 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260806 00:54:15.185316 4755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.175826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44336 (local address 127.27.77.252:43643)
0806 00:54:15.175944 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.175950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.175967 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:54:15.176080 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.176086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.176159 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.176257 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.176744 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.177606 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.178824 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.179217 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.179327 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.181799 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.181803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.181806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.181832 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.183452 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.184028 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.184031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.184034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.184118 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.184442 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.184445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.184447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.184651 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.184747 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.185046 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.185163 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:54:15.186750 3780 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.27.77.252:43643: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e239ce00299a4509830746a4126c44cf"
member_type: NON_VOTER
last_known_addr {
host: "127.27.77.252"
port: 43643
}
attrs {
promote: true
}
}
dest_uuid: "e9a22350b1e44dcc935e18a905203e01"
cas_config_opid_index: 5
I20260806 00:54:15.187393 3780 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } }
I20260806 00:54:15.188499 4542 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e239ce00299a4509830746a4126c44cf (127.27.77.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } } }
I20260806 00:54:15.190024 4542 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.27.77.252:43643
I20260806 00:54:15.190210 4542 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } } }
I20260806 00:54:15.190315 4542 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:15.191517 4716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:15.192961 4716 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 1 FOLLOWER]: Refusing update from remote peer e9a22350b1e44dcc935e18a905203e01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 00:54:15.193555 4543 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:15.195899 4538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.190659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:45881 (local address 127.0.0.1:53558)
0806 00:54:15.191160 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.191246 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.191356 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.191834 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.191837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.191854 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:15.191936 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.191942 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.192989 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.192992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.194189 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.194202 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.195301 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.195305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.195393 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.195630 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.195706 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.195777 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427}
I20260806 00:54:15.196293 4756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.190758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53558 (local address 127.27.77.253:45881)
0806 00:54:15.190878 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.190882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.191452 (+ 570us) server_negotiation.cc:407] Connection header received
0806 00:54:15.191508 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.191511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.191612 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.191703 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:15.192063 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.192850 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.194342 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.194978 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.195081 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.195235 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.196102 (+ 867us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.196181 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:54:15.196722 4605 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 LEARNER]: Refusing update from remote peer e9a22350b1e44dcc935e18a905203e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 00:54:15.197153 4543 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 00:54:15.197808 4551 master.cc:537] Master@127.27.77.252:43643 shutting down...
I20260806 00:54:15.199082 4605 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e239ce00299a4509830746a4126c44cf (127.27.77.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } } }
I20260806 00:54:15.201212 4791 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } } }
I20260806 00:54:15.201345 4791 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: This master's current role is: LEARNER
I20260806 00:54:15.201625 4642 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9a22350b1e44dcc935e18a905203e01. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } } }
I20260806 00:54:15.201752 4642 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: This master's current role is: LEARNER
I20260806 00:54:15.203840 4716 raft_consensus.cc:858] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } }
W20260806 00:54:15.203949 4716 raft_consensus.cc:1565] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } }
I20260806 00:54:15.205307 4716 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bbdf6367d78047ddafb01c62630ff258 (127.27.77.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } } }
I20260806 00:54:15.206391 4789 mvcc.cc:205] Tried to move back new op lower bound from 7315364469583216640 to 7315364469442969600. Current Snapshot: MvccSnapshot[applied={T|T < 7315364469480656896}]
I20260806 00:54:15.207340 4759 sys_catalog.cc:455] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9a22350b1e44dcc935e18a905203e01. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } } }
I20260806 00:54:15.207445 4759 sys_catalog.cc:458] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 00:54:15.207554 4759 sys_catalog.cc:455] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } } }
I20260806 00:54:15.207633 4759 sys_catalog.cc:458] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 00:54:15.207698 4759 sys_catalog.cc:455] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: NON_VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: true } } }
I20260806 00:54:15.207751 4759 sys_catalog.cc:458] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 00:54:15.208098 4543 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01: attempting to promote NON_VOTER bbdf6367d78047ddafb01c62630ff258 to VOTER
I20260806 00:54:15.208506 4543 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } }
I20260806 00:54:15.209794 4605 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 LEARNER]: Refusing update from remote peer e9a22350b1e44dcc935e18a905203e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 00:54:15.210299 4542 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260806 00:54:15.210503 3763 consensus_peers.cc:487] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 -> Peer e239ce00299a4509830746a4126c44cf (127.27.77.252:43643): Couldn't send request to peer e239ce00299a4509830746a4126c44cf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 00:54:15.212507 4542 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bbdf6367d78047ddafb01c62630ff258 (127.27.77.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } } }
I20260806 00:54:15.213909 4542 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } } }
I20260806 00:54:15.214022 4542 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:15.214135 4542 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } } }
I20260806 00:54:15.214207 4542 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:15.214824 4605 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bbdf6367d78047ddafb01c62630ff258 (127.27.77.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } } }
I20260806 00:54:15.216751 4642 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } } }
I20260806 00:54:15.216863 4642 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:54:15.218253 4551 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260806 00:54:15.218562 4551 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260806 00:54:15.218664 4551 tablet_replica.cc:330] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: stopping tablet replica
I20260806 00:54:15.224371 4551 master.cc:552] Master@127.27.77.252:43643 shutdown complete.
I20260806 00:54:15.224468 4551 master_runner.cc:305] Clearing existing system tablet
I20260806 00:54:15.224764 4551 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:15.224872 4551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:15.226392 4551 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 00:54:15.227409 4804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:15.227670 4551 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:15.227953 4551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 00:54:15.228039 4551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/wal
uuid: "e239ce00299a4509830746a4126c44cf"
format_stamp: "Formatted at 2026-08-06 00:54:15 on dist-test-slave-sb2z"
server_key: "658c939cc1d2cb95f777b30e76e685ea"
server_key_iv: "034242b0aba43a5cbc30133707205615"
server_key_version: "encryptionkey@0"
I20260806 00:54:15.229862 4551 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:15.232460 4551 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 00:54:15.232625 4551 log.cc:1196] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 00:54:15.233008 4551 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: Deleting consensus metadata
I20260806 00:54:15.233227 4551 master_runner.cc:315] Copying system tablet from 127.27.77.254:38937
I20260806 00:54:15.234444 4551 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: tablet copy: Beginning tablet copy session from remote peer at address 127.27.77.254:38937
I20260806 00:54:15.243999 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.235176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42938 (local address 127.27.77.254:38937)
0806 00:54:15.235308 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.235312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.235552 (+ 240us) server_negotiation.cc:407] Connection header received
0806 00:54:15.235689 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.235692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.235745 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.235840 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.236376 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.237219 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.238421 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.238737 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.238839 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.240385 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.240389 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.240392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.240420 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.242157 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.242654 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.242658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.242660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.242748 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.243054 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.243057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.243059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.243273 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.243373 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.243703 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.243826 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:54:15.244006 4811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.235085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.0.0.1:42938)
0806 00:54:15.235415 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.235431 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.235573 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.235882 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.235885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.236119 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.236242 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.236249 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.237350 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.237354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.238296 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.238308 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.238864 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.238868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.238993 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.239760 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.239776 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.240212 (+ 436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.242282 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.242286 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.242288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.242548 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.242851 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.242854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.242856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.242953 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.243379 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.243382 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.243486 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.243644 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.243796 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":93,"threads_started":1}
I20260806 00:54:15.244705 3806 tablet_copy_service.cc:140] P e9a22350b1e44dcc935e18a905203e01: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e239ce00299a4509830746a4126c44cf ({username='oryx', principal='oryx/127.27.77.252@KRBTEST.COM'} at 127.0.0.1:42938)
I20260806 00:54:15.244801 3806 tablet_copy_service.cc:161] P e9a22350b1e44dcc935e18a905203e01: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e239ce00299a4509830746a4126c44cf at {username='oryx', principal='oryx/127.27.77.252@KRBTEST.COM'} at 127.0.0.1:42938: session id = e239ce00299a4509830746a4126c44cf-00000000000000000000000000000000
I20260806 00:54:15.245905 3806 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 00:54:15.247363 4551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 00:54:15.249840 4551 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: tablet copy: Starting download of 0 data blocks...
I20260806 00:54:15.250044 4551 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: tablet copy: Starting download of 1 WAL segments...
I20260806 00:54:15.251617 4551 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 00:54:15.253539 3806 tablet_copy_service.cc:342] P e9a22350b1e44dcc935e18a905203e01: Request end of tablet copy session e239ce00299a4509830746a4126c44cf-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.27.77.252@KRBTEST.COM'} at 127.0.0.1:42938
I20260806 00:54:15.253631 3806 tablet_copy_service.cc:433] P e9a22350b1e44dcc935e18a905203e01: ending tablet copy session e239ce00299a4509830746a4126c44cf-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e239ce00299a4509830746a4126c44cf
I20260806 00:54:15.261814 4551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:15.264496 4812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:15.264863 4551 server_base.cc:1034] running on GCE node
W20260806 00:54:15.265007 4815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:15.268045 4813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:15.268313 4551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:15.268705 4551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:15.269862 4551 hybrid_clock.cc:648] HybridClock initialized: now 1785977655269827 us; error 54 us; skew 500 ppm
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:15.271639 4551 init.cc:377] Logged in from keytab as oryx/127.27.77.252@KRBTEST.COM (short username oryx)
I20260806 00:54:15.272127 4551 webserver.cc:460] Webserver started at http://127.27.77.252:41567/ using document root <none> and password file <none>
I20260806 00:54:15.272354 4551 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:15.272396 4551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:15.276152 4213 heartbeater.cc:503] Master 127.27.77.254:38937 was elected leader, sending a full tablet report...
I20260806 00:54:15.279383 4551 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260806 00:54:15.280812 4826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:15.281055 4551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 00:54:15.281124 4551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/wal
uuid: "e239ce00299a4509830746a4126c44cf"
format_stamp: "Formatted at 2026-08-06 00:54:15 on dist-test-slave-sb2z"
server_key: "658c939cc1d2cb95f777b30e76e685ea"
server_key_iv: "034242b0aba43a5cbc30133707205615"
server_key_version: "encryptionkey@0"
I20260806 00:54:15.281219 4551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:15.298023 4551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:15.300432 4551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:15.307188 4551 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.252:43643
I20260806 00:54:15.307232 4898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.252:43643 every 8 connection(s)
I20260806 00:54:15.308495 4551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/master-2/data/info.pb
I20260806 00:54:15.310567 4899 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:54:15.311793 4899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: Bootstrap starting.
I20260806 00:54:15.323756 4899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:54:15.324234 4899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: Bootstrap complete.
I20260806 00:54:15.324985 4899 raft_consensus.cc:348] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } }
I20260806 00:54:15.325228 4899 raft_consensus.cc:729] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e239ce00299a4509830746a4126c44cf, State: Initialized, Role: LEARNER
I20260806 00:54:15.325397 4899 consensus_queue.cc:260] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } }
I20260806 00:54:15.325887 4907 sys_catalog.cc:455] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } } }
I20260806 00:54:15.326001 4899 sys_catalog.cc:564] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:15.326021 4907 sys_catalog.cc:458] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: This master's current role is: LEARNER
I20260806 00:54:15.329370 4919 catalog_manager.cc:1259] Loaded cluster ID: 382f6c59a82144088da901f2ccc4a5ad
I20260806 00:54:15.329425 4919 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: loading cluster ID for follower catalog manager: success
W20260806 00:54:15.328971 4786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:15.330793 4786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:15.330862 4786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 00:54:15.331730 4919 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: acquiring CA information for follower catalog manager: success
I20260806 00:54:15.333053 4919 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 00:54:15.336956 4786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:15.337045 4786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:15.337167 4786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:54:15.344718 4786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.193.keytab
--principal=oryx/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:39337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=33359
--tserver_master_addrs=127.27.77.254:38937,127.27.77.253:45881,127.27.77.252:43643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:15.346033 4786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:15.347467 4786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:15.362819 4923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:15.363057 4786 server_base.cc:1034] running on GCE node
W20260806 00:54:15.362874 4925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:15.363312 4922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:15.363849 4786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:15.364573 4786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:15.365780 4786 hybrid_clock.cc:648] HybridClock initialized: now 1785977655365763 us; error 33 us; skew 500 ppm
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:15.368769 4786 init.cc:377] Logged in from keytab as oryx/127.27.77.193@KRBTEST.COM (short username oryx)
I20260806 00:54:15.369725 4786 webserver.cc:460] Webserver started at http://127.27.77.193:33359/ using document root <none> and password file <none>
I20260806 00:54:15.370316 4786 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:15.370433 4786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:15.374867 4786 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:54:15.377210 4934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:15.378818 4786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 00:54:15.378968 4786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda"
format_stamp: "Formatted at 2026-08-06 00:54:13 on dist-test-slave-sb2z"
server_key: "2a8547107e6f81b0f2e0100c73f508c2"
server_key_iv: "4a0c3a9abc41ae727b3d9877f2c1a611"
server_key_version: "encryptionkey@0"
I20260806 00:54:15.379653 4786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:15.411970 4786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:15.415513 4786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:15.415813 4786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:15.416554 4786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:15.417484 4786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:15.417558 4786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:15.417632 4786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:15.417683 4786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:15.428329 4786 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:39337
I20260806 00:54:15.428382 5081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:39337 every 8 connection(s)
I20260806 00:54:15.429447 4786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:54:15.439283 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 4786
I20260806 00:54:15.439513 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 4048
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.253@KRBTEST.COM
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.252@KRBTEST.COM
I20260806 00:54:15.448791 4756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.432999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:54317 (local address 127.27.77.253:45881)
0806 00:54:15.433162 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.433167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.433184 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:54:15.433868 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.433872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.433930 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.433999 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.435447 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.436051 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.437755 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.438009 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.438130 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.443907 (+ 5777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.443911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.443913 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.443945 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.445788 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.446622 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.446626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.446628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.446729 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.447335 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.447339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.447342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.447637 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.447712 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.448483 (+ 771us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.448617 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:54:15.449368 5087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.431335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:52745 (local address 127.27.77.252:43643)
0806 00:54:15.431713 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.431717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.432678 (+ 961us) server_negotiation.cc:407] Connection header received
0806 00:54:15.434140 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.434144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.434201 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.434293 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.435457 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.436545 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.437512 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.437844 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.437941 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.444343 (+ 6402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.444346 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.444349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.444385 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.446753 (+ 2368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.447458 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.447462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.447463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.447579 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.447902 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.447905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.447906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.448124 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.448220 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.449060 (+ 840us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.449210 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":90,"threads_started":1}
I20260806 00:54:15.449932 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.431418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:42299 (local address 127.27.77.254:38937)
0806 00:54:15.431597 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.431604 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.433494 (+ 1890us) server_negotiation.cc:407] Connection header received
0806 00:54:15.433882 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.433886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.433949 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.434010 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.435461 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.436299 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.437724 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.438045 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.438144 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.443664 (+ 5520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.443672 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.443679 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.443726 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.446565 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.447454 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.447461 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.447466 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.447639 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.448189 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.448196 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.448200 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.448529 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.448643 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.449468 (+ 825us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.449663 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:54:15.450237 5089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.431890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:43643 (local address 127.27.77.193:52745)
0806 00:54:15.432494 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.432547 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.433644 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.434328 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.434331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.434782 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.435297 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.435305 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.436662 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.436667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.437380 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.437389 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.438130 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.438134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.438354 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.439247 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.439265 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.444165 (+ 4900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.446871 (+ 2706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.446873 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.446875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.447328 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.447697 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.447700 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.447702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.447795 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.448265 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.448267 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.448349 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.448483 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.448849 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":130,"spinlock_wait_cycles":27136,"thread_start_us":608,"threads_started":1}
I20260806 00:54:15.450517 5090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.431890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:45881 (local address 127.27.77.193:54317)
0806 00:54:15.432834 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.432849 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.433643 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.434228 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.434234 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.434720 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.435283 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.435298 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.436215 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.436221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.437598 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.437610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.438135 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.438139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.438346 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.438972 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.438998 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.443755 (+ 4757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.446042 (+ 2287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.446049 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.446071 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.446483 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.446857 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.446861 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.446863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.447207 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.447795 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.447802 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.448179 (+ 377us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.448980 (+ 801us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.449126 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":193,"spinlock_wait_cycles":108800,"thread_start_us":174,"threads_started":1}
I20260806 00:54:15.451799 5092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.433112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.193:42299)
0806 00:54:15.433368 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.433382 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.433644 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.434225 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.434232 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.434785 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.435295 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.435305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.436423 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.436427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.437598 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.437610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.438292 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.438296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.438398 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.439326 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.439342 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.443392 (+ 4050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.446753 (+ 3361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.446760 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.446765 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.447296 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.447902 (+ 606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.447908 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.447913 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.448043 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.448742 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.448748 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.449196 (+ 448us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.451210 (+ 2014us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.451434 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":168,"spinlock_wait_cycles":61056,"thread_start_us":85,"threads_started":1}
I20260806 00:54:15.452025 5085 heartbeater.cc:347] Connected to a master server at 127.27.77.253:45881
I20260806 00:54:15.452541 5085 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:15.453135 5085 heartbeater.cc:511] Master 127.27.77.253:45881 requested a full tablet report, sending...
I20260806 00:54:15.454810 4595 ts_manager.cc:194] Registered new tserver with Master: 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda (127.27.77.193:39337)
I20260806 00:54:15.459815 5083 heartbeater.cc:347] Connected to a master server at 127.27.77.252:43643
I20260806 00:54:15.459942 5083 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:15.460203 5083 heartbeater.cc:511] Master 127.27.77.252:43643 requested a full tablet report, sending...
I20260806 00:54:15.460518 5084 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38937
I20260806 00:54:15.460652 5084 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:15.460851 5084 heartbeater.cc:511] Master 127.27.77.254:38937 requested a full tablet report, sending...
I20260806 00:54:15.461688 4836 ts_manager.cc:194] Registered new tserver with Master: 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda (127.27.77.193:39337)
I20260806 00:54:15.461762 3780 ts_manager.cc:194] Re-registered known tserver with Master: 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda (127.27.77.193:39337)
I20260806 00:54:15.462703 3780 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:42299
I20260806 00:54:15.468236 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:44163
--local_ip_for_outbound_sockets=127.27.77.194
--tserver_master_addrs=127.27.77.254:38937
--webserver_port=37613
--webserver_interface=127.27.77.194
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.194.keytab
--principal=oryx/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.77.254:38937,127.27.77.253:45881
--tserver_master_addrs=127.27.77.254:38937,127.27.77.253:45881,127.27.77.252:43643 with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:54:15.510345 4406 heartbeater.cc:503] Master 127.27.77.254:38937 was elected leader, sending a full tablet report...
W20260806 00:54:15.588424 5094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:15.588701 5094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:15.588747 5094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:15.592718 5094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:15.592779 5094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:15.592840 5094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:54:15.598491 5094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.194.keytab
--principal=oryx/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:44163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=37613
--tserver_master_addrs=127.27.77.254:38937,127.27.77.253:45881,127.27.77.252:43643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:15.599664 5094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:15.600683 5094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:15.610805 5100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:15.610838 5101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:15.611099 5094 server_base.cc:1034] running on GCE node
W20260806 00:54:15.611212 5105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:15.611703 5094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:15.612366 5094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:15.613600 5094 hybrid_clock.cc:648] HybridClock initialized: now 1785977655613566 us; error 46 us; skew 500 ppm
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:15.617256 5094 init.cc:377] Logged in from keytab as oryx/127.27.77.194@KRBTEST.COM (short username oryx)
I20260806 00:54:15.618178 5094 webserver.cc:460] Webserver started at http://127.27.77.194:37613/ using document root <none> and password file <none>
I20260806 00:54:15.618774 5094 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:15.618880 5094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:15.622989 5094 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:54:15.625138 5121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:15.626514 5094 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:54:15.626703 5094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "e3096c6aa5f340f58b7d3b0a68711297"
format_stamp: "Formatted at 2026-08-06 00:54:14 on dist-test-slave-sb2z"
server_key: "99f9452295105ab8121a3cd5296821e2"
server_key_iv: "9e5e59d615bb89a61ac554219bbbd37a"
server_key_version: "encryptionkey@0"
I20260806 00:54:15.627238 5094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:15.645596 5094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:15.649251 5094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:15.649478 5094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:15.650091 5094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:15.650983 5094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:15.651062 5094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:15.651158 5094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:15.651211 5094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:15.661557 5094 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:44163
I20260806 00:54:15.661644 5281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:44163 every 8 connection(s)
I20260806 00:54:15.662812 5094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:54:15.663564 4538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.658774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:43643 (local address 127.0.0.1:44348)
0806 00:54:15.659029 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.659047 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.659156 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.659697 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.659704 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.659730 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:15.659857 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.659869 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.660859 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.660866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.662091 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.662105 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.663130 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.663133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.663221 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.663244 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.663310 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.663394 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260806 00:54:15.663575 5087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.658947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44348 (local address 127.27.77.252:43643)
0806 00:54:15.659178 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.659185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.659210 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:54:15.659284 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.659288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.659371 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.659496 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:15.660039 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.660727 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.662298 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.662986 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.663131 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.663303 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.663342 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.663419 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260806 00:54:15.664196 4859 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 00:54:15.664491 4907 sys_catalog.cc:455] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9a22350b1e44dcc935e18a905203e01. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } } }
I20260806 00:54:15.664649 4907 sys_catalog.cc:458] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: This master's current role is: LEARNER
I20260806 00:54:15.666261 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 5094
I20260806 00:54:15.666458 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 4222
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.253@KRBTEST.COM
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.252@KRBTEST.COM
I20260806 00:54:15.684084 4756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.665992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:58013 (local address 127.27.77.253:45881)
0806 00:54:15.666131 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.666136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.670346 (+ 4210us) server_negotiation.cc:407] Connection header received
0806 00:54:15.670466 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.670471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.670527 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.670608 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.672293 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.673099 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.674881 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.675237 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.675362 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.678591 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.678595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.678597 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.678627 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.680705 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.681345 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.681349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.681350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.681445 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.681887 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.681894 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.681899 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.682780 (+ 881us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.682887 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.683728 (+ 841us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.683862 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 00:54:15.685449 5087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.665861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:47927 (local address 127.27.77.252:43643)
0806 00:54:15.665966 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.665970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.670695 (+ 4725us) server_negotiation.cc:407] Connection header received
0806 00:54:15.670869 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.670872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.670919 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.670982 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.672645 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.673370 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.676863 (+ 3493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.677239 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.677326 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.681649 (+ 4323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.681653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.681656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.681685 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.683414 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.684014 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.684018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.684019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.684106 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.684427 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.684431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.684442 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.684693 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.684776 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.685174 (+ 398us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.685293 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:54:15.686416 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.666352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:45881 (local address 127.27.77.194:58013)
0806 00:54:15.670240 (+ 3888us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.670251 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.670375 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.670955 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.670968 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.671603 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.672136 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.672151 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.673572 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.673575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.674748 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.674759 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.675402 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.675407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.675687 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.676360 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.676381 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.678367 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.680846 (+ 2479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.680852 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.680869 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.681232 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.681559 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.681564 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.681569 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.681761 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.682982 (+ 1221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.682989 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.683403 (+ 414us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.684657 (+ 1254us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.685080 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3803,"mutex_wait_us":69,"spinlock_wait_cycles":39680,"thread_start_us":186,"threads_started":1}
I20260806 00:54:15.686416 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.668863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:43643 (local address 127.27.77.194:47927)
0806 00:54:15.670614 (+ 1751us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.670628 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.670762 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.671128 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.671133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.672378 (+ 1245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.672502 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.672509 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.674920 (+ 2411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.674923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.676712 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.676724 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.677504 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.677507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.677615 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.678291 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.678308 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.681488 (+ 3180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.683607 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.683612 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.683614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.683897 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.684233 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.684238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.684241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.684326 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.684860 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.684862 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.684975 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.685606 (+ 631us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.685745 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1666,"mutex_wait_us":2,"thread_start_us":878,"threads_started":1}
I20260806 00:54:15.689297 5283 heartbeater.cc:347] Connected to a master server at 127.27.77.252:43643
I20260806 00:54:15.689581 5283 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:15.690243 5283 heartbeater.cc:511] Master 127.27.77.252:43643 requested a full tablet report, sending...
I20260806 00:54:15.691200 5288 heartbeater.cc:347] Connected to a master server at 127.27.77.253:45881
I20260806 00:54:15.691299 5288 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:15.691510 5288 heartbeater.cc:511] Master 127.27.77.253:45881 requested a full tablet report, sending...
I20260806 00:54:15.691726 4836 ts_manager.cc:194] Registered new tserver with Master: e3096c6aa5f340f58b7d3b0a68711297 (127.27.77.194:44163)
I20260806 00:54:15.692373 4595 ts_manager.cc:194] Registered new tserver with Master: e3096c6aa5f340f58b7d3b0a68711297 (127.27.77.194:44163)
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
I20260806 00:54:15.694626 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:42489
--local_ip_for_outbound_sockets=127.27.77.195
--tserver_master_addrs=127.27.77.254:38937
--webserver_port=39367
--webserver_interface=127.27.77.195
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.195.keytab
--principal=oryx/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.77.254:38937,127.27.77.253:45881
--tserver_master_addrs=127.27.77.254:38937,127.27.77.253:45881,127.27.77.252:43643 with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:54:15.696581 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.666352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.194:47513)
0806 00:54:15.668725 (+ 2373us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.668787 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.669902 (+ 1115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.686530 (+ 16628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.686537 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.686765 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.686917 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.686928 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.689173 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.689176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.689886 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.689897 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.690539 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.690542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.690617 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.691138 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.691149 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.692888 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.694817 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.694823 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.694828 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.695093 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.695382 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.695388 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.695393 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.695496 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.695904 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.695910 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.696051 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.696176 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.696351 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2144,"thread_start_us":1973,"threads_started":1}
I20260806 00:54:15.697050 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.668742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:47513 (local address 127.27.77.254:38937)
0806 00:54:15.685848 (+ 17106us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.685855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.685875 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:54:15.685925 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.685931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.685996 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.686129 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.688126 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.688998 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.690040 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.690282 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.690409 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.693037 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.693041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.693044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.693079 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.694659 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.695201 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.695204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.695206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.695286 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.695623 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.695626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.695628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.695807 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.695884 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.696826 (+ 942us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.696941 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16976}
I20260806 00:54:15.697641 5287 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38937
I20260806 00:54:15.697748 5287 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:15.697945 5287 heartbeater.cc:511] Master 127.27.77.254:38937 requested a full tablet report, sending...
I20260806 00:54:15.698619 3780 ts_manager.cc:194] Re-registered known tserver with Master: e3096c6aa5f340f58b7d3b0a68711297 (127.27.77.194:44163)
I20260806 00:54:15.699209 3780 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:47513
W20260806 00:54:15.817955 5295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:15.818267 5295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:15.818398 5295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:15.823206 5295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:15.823293 5295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:15.823392 5295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:54:15.829819 5295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.195.keytab
--principal=oryx/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:42489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=39367
--tserver_master_addrs=127.27.77.254:38937,127.27.77.253:45881,127.27.77.252:43643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:15.831174 5295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:15.832423 5295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:15.843611 5302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:15.843644 5303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:15.843648 5306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:15.843873 5295 server_base.cc:1034] running on GCE node
I20260806 00:54:15.844322 5295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:15.845019 5295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:15.846231 5295 hybrid_clock.cc:648] HybridClock initialized: now 1785977655846197 us; error 48 us; skew 500 ppm
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:15.849643 5295 init.cc:377] Logged in from keytab as oryx/127.27.77.195@KRBTEST.COM (short username oryx)
I20260806 00:54:15.850744 5295 webserver.cc:460] Webserver started at http://127.27.77.195:39367/ using document root <none> and password file <none>
I20260806 00:54:15.851389 5295 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:15.851575 5295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:15.856010 5295 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 00:54:15.858227 5314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:15.859463 5295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:54:15.859653 5295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "cb3523d7e6f84f349d3bb5c864757290"
format_stamp: "Formatted at 2026-08-06 00:54:14 on dist-test-slave-sb2z"
server_key: "4e09793bf0ba21f9f308d86dacae935c"
server_key_iv: "4716c018ac852736dcf9fbcafadb69d4"
server_key_version: "encryptionkey@0"
I20260806 00:54:15.860278 5295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:15.881534 5295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:15.884967 5295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:15.885185 5295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:15.885821 5295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:15.886787 5295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:15.886860 5295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:15.886933 5295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:15.886986 5295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:15.897449 5295 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:42489
I20260806 00:54:15.897480 5467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:42489 every 8 connection(s)
I20260806 00:54:15.898586 5295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:54:15.903023 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 5295
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.252@KRBTEST.COM
I20260806 00:54:15.917896 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.900476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:36699 (local address 127.27.77.254:38937)
0806 00:54:15.902754 (+ 2278us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.902759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.902789 (+ 30us) server_negotiation.cc:407] Connection header received
0806 00:54:15.903444 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.903448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.903511 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.903654 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.906024 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.906785 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.908214 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.908547 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.908653 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.912310 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.912314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.912317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.912347 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.915238 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.915958 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.915962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.915964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.916065 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.916481 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.916486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.916489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.916766 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.916876 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.917538 (+ 662us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.917688 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2182,"spinlock_wait_cycles":1792}
Aug 06 00:54:15 dist-test-slave-sb2z krb5kdc[3718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977655, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.253@KRBTEST.COM
I20260806 00:54:15.919297 5475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.900952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.195:36699)
0806 00:54:15.902171 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.902235 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.903216 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.904300 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.904304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.905170 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.905893 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.905904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.906890 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.906896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.908098 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.908110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.908996 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.908999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.909213 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.909879 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.909898 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.912131 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.915387 (+ 3256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.915394 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.915429 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.915804 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.916177 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.916180 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.916182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.916371 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.916889 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.916895 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.917237 (+ 342us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.917442 (+ 205us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.917810 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1027,"mutex_wait_us":600,"spinlock_wait_cycles":43776,"thread_start_us":187,"threads_started":1}
I20260806 00:54:15.920725 5486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.905555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53560 (local address 127.27.77.253:45881)
0806 00:54:15.908760 (+ 3205us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.908765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.908778 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:54:15.908812 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.908815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.908864 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.908950 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.910050 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.910769 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.912635 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.913046 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.913179 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.915128 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.915132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.915135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.915161 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.917467 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.918300 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.918305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.918307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.918400 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.918722 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.918725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.918727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.918956 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.919307 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.920382 (+ 1075us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.920525 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3120,"thread_start_us":80,"threads_started":1}
I20260806 00:54:15.920755 5469 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38937
I20260806 00:54:15.921056 5469 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:15.921710 5469 heartbeater.cc:511] Master 127.27.77.254:38937 requested a full tablet report, sending...
I20260806 00:54:15.923570 3780 ts_manager.cc:194] Re-registered known tserver with Master: cb3523d7e6f84f349d3bb5c864757290 (127.27.77.195:42489)
I20260806 00:54:15.923825 5487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.905568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44364 (local address 127.27.77.252:43643)
0806 00:54:15.908268 (+ 2700us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.908272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.911208 (+ 2936us) server_negotiation.cc:407] Connection header received
0806 00:54:15.911407 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.911410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.911463 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.911562 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.913337 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.914105 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.915334 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.915778 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.915873 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.918660 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.918666 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.918669 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.918704 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.920690 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.921718 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.921722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.921724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.921846 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.922172 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.922175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.922177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.922419 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.922554 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.923360 (+ 806us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.923484 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2623,"thread_start_us":82,"threads_started":1}
I20260806 00:54:15.924467 3780 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:36699
I20260806 00:54:15.926402 5087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.900571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:42889 (local address 127.27.77.252:43643)
0806 00:54:15.900767 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.900772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.902344 (+ 1572us) server_negotiation.cc:407] Connection header received
0806 00:54:15.903673 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.903676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.903729 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.903786 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.905688 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.906462 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.908397 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.908757 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.908829 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.922559 (+ 13730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.922563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.922565 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.922593 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.924477 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.924996 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.925000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.925002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.925090 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.925437 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.925440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.925441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.925686 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.925776 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.926115 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.926253 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 00:54:15.926888 4756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.900994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:50273 (local address 127.27.77.253:45881)
0806 00:54:15.901127 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.901132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.902338 (+ 1206us) server_negotiation.cc:407] Connection header received
0806 00:54:15.903807 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.903810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.903864 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.903922 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.906940 (+ 3018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.907756 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.910004 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.910340 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.910412 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.922555 (+ 12143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.922559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.922562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.922590 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.924889 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.925483 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.925488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.925491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.925567 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.925885 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.925889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.925890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.926131 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.926202 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.926637 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.926747 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:54:15.927086 5478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.901213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:43643 (local address 127.27.77.195:42889)
0806 00:54:15.902222 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.902237 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.903506 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.904111 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.904119 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.904787 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.905495 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.905515 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.906680 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.906683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.908289 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.908295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.909559 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.909563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.909663 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.912705 (+ 3042us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.912721 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.916762 (+ 4041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.924602 (+ 7840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.924606 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.924608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.924884 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.925224 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.925226 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.925227 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.925345 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.925820 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.925826 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.925926 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.926747 (+ 821us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.926893 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"mutex_wait_us":351,"spinlock_wait_cycles":108800,"thread_start_us":105,"threads_started":1}
I20260806 00:54:15.927436 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.900952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:45881 (local address 127.27.77.195:50273)
0806 00:54:15.902172 (+ 1220us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.902235 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.903217 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.904111 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.904120 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.904735 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:15.906077 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.906084 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.907891 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.907895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.909855 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.909867 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.910580 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.910584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.911874 (+ 1290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.917929 (+ 6055us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:15.917943 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:15.922408 (+ 4465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:15.925006 (+ 2598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.925009 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.925011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.925253 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.925684 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:15.925687 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:15.925690 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:15.925778 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:15.926289 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:15.926292 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:15.926421 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.927019 (+ 598us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.927227 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"mutex_wait_us":539,"spinlock_wait_cycles":577536,"thread_start_us":704,"threads_started":1}
I20260806 00:54:15.927935 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.906449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42944 (local address 127.27.77.254:38937)
0806 00:54:15.912949 (+ 6500us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.912954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.913694 (+ 740us) server_negotiation.cc:407] Connection header received
0806 00:54:15.913740 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.913744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.913806 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.913897 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:15.914951 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.915849 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.917023 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.917338 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.917441 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.920122 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:15.920126 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:15.920129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:15.920161 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:15.921959 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.922574 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.922578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.922580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.922675 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:15.925525 (+ 2850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:15.925529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:15.925530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:15.925747 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:15.925910 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.927605 (+ 1695us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.927741 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6414,"spinlock_wait_cycles":9600,"thread_start_us":73,"threads_started":1}
I20260806 00:54:15.928275 5473 heartbeater.cc:347] Connected to a master server at 127.27.77.253:45881
I20260806 00:54:15.928375 5473 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:15.928439 5468 heartbeater.cc:347] Connected to a master server at 127.27.77.252:43643
I20260806 00:54:15.928542 5468 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:15.928611 5473 heartbeater.cc:511] Master 127.27.77.253:45881 requested a full tablet report, sending...
I20260806 00:54:15.928651 5468 heartbeater.cc:511] Master 127.27.77.252:43643 requested a full tablet report, sending...
I20260806 00:54:15.929386 4836 ts_manager.cc:194] Registered new tserver with Master: cb3523d7e6f84f349d3bb5c864757290 (127.27.77.195:42489)
I20260806 00:54:15.929375 4595 ts_manager.cc:194] Registered new tserver with Master: cb3523d7e6f84f349d3bb5c864757290 (127.27.77.195:42489)
I20260806 00:54:15.931206 3780 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:54:15.934307 3780 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:54:15.936950 5496 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: NON_VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: true } } }
I20260806 00:54:15.937129 5496 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:15.939065 5496 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01: attempting to promote NON_VOTER e239ce00299a4509830746a4126c44cf to VOTER
I20260806 00:54:15.939462 5496 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } }
I20260806 00:54:15.940785 4859 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 1 LEARNER]: Refusing update from remote peer e9a22350b1e44dcc935e18a905203e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 00:54:15.940802 4605 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 FOLLOWER]: Refusing update from remote peer e9a22350b1e44dcc935e18a905203e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 00:54:15.941227 5496 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 00:54:15.941394 4801 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 00:54:15.943248 4801 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e239ce00299a4509830746a4126c44cf (127.27.77.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } } }
I20260806 00:54:15.943890 4605 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e239ce00299a4509830746a4126c44cf (127.27.77.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } } }
I20260806 00:54:15.943886 4859 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e239ce00299a4509830746a4126c44cf (127.27.77.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } } }
I20260806 00:54:15.944908 4801 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } } }
I20260806 00:54:15.945026 4801 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:15.945181 4801 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } } }
I20260806 00:54:15.945302 4801 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:15.945431 4801 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } } }
I20260806 00:54:15.945513 4801 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:15.945734 4907 sys_catalog.cc:455] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } } }
I20260806 00:54:15.945858 4907 sys_catalog.cc:458] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:54:15.946519 5505 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9a22350b1e44dcc935e18a905203e01" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } } }
I20260806 00:54:15.946667 5505 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:54:15.959815 5508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.954538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:39337 (local address 127.0.0.1:60742)
0806 00:54:15.954789 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.954804 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.954898 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.955511 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.955516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.955573 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:15.955781 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.955792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.956978 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.956981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.958205 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.958217 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.959322 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.959328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.959437 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.959467 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.959577 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.959653 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":74,"spinlock_wait_cycles":3328,"thread_start_us":123,"threads_started":1}
I20260806 00:54:15.960132 5510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.954701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60742 (local address 127.27.77.193:39337)
0806 00:54:15.955034 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.955040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.955058 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:54:15.955121 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.955126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.955341 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.955547 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:15.955893 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.956886 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.958345 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.959198 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.959313 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.959493 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.959622 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.959739 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":60,"threads_started":1}
I20260806 00:54:15.960527 4538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.954051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:42489 (local address 127.0.0.1:42734)
0806 00:54:15.954184 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.954199 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.954279 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.955546 (+ 1267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.955549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.955578 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:15.955747 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.955755 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.957316 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.957319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.958440 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.958449 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.960055 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.960058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.960162 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.960194 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.960300 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.960395 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":60,"spinlock_wait_cycles":1280}
I20260806 00:54:15.960605 5506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.954275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42734 (local address 127.27.77.195:42489)
0806 00:54:15.954976 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.954990 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.955015 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:54:15.955107 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.955113 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.955341 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.955896 (+ 555us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:15.956072 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.957188 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.958578 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.959938 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.960061 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.960216 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.960323 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.960456 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":168,"threads_started":1}
I20260806 00:54:15.961697 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.954558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:44163 (local address 127.0.0.1:36848)
0806 00:54:15.955998 (+ 1440us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:15.956013 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:15.956109 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:15.956679 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:15.956682 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:15.956701 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:15.956780 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.956785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.958048 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.958054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.959070 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:15.959083 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.961337 (+ 2254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.961340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:15.961424 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.961445 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:54:15.961510 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 00:54:15.961575 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1169,"thread_start_us":103,"threads_started":1}
I20260806 00:54:15.961799 5511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:15.954988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36848 (local address 127.27.77.194:44163)
0806 00:54:15.955316 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:15.955327 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:15.956215 (+ 888us) server_negotiation.cc:407] Connection header received
0806 00:54:15.956305 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:15.956310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:15.956517 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:15.956677 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:15.956878 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.957929 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.959197 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:15.961199 (+ 2002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:15.961338 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:15.961479 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:15.961536 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 00:54:15.961642 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":135,"threads_started":1}
I20260806 00:54:15.963517 4991 tablet_service.cc:1467] Processing CreateTablet for tablet 4be920ffcaae49da80a09a24338f3863 (DEFAULT_TABLE table=test-table [id=a2cf9cb58a5c41e396988de3edf8e56e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:15.963744 5382 tablet_service.cc:1467] Processing CreateTablet for tablet 4be920ffcaae49da80a09a24338f3863 (DEFAULT_TABLE table=test-table [id=a2cf9cb58a5c41e396988de3edf8e56e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:15.964792 5382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4be920ffcaae49da80a09a24338f3863. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:15.964792 4991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4be920ffcaae49da80a09a24338f3863. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:15.964430 5189 tablet_service.cc:1467] Processing CreateTablet for tablet 4be920ffcaae49da80a09a24338f3863 (DEFAULT_TABLE table=test-table [id=a2cf9cb58a5c41e396988de3edf8e56e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:15.965384 5189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4be920ffcaae49da80a09a24338f3863. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:15.973318 5512 tablet_bootstrap.cc:492] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297: Bootstrap starting.
I20260806 00:54:15.973695 5513 tablet_bootstrap.cc:492] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: Bootstrap starting.
I20260806 00:54:15.974341 5514 tablet_bootstrap.cc:492] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290: Bootstrap starting.
I20260806 00:54:15.975975 5512 tablet_bootstrap.cc:654] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:15.976681 5514 tablet_bootstrap.cc:654] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:15.976794 5512 log.cc:824] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:15.977610 5513 tablet_bootstrap.cc:654] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:15.977699 5514 log.cc:824] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:15.978500 5513 log.cc:824] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:15.979259 5512 tablet_bootstrap.cc:492] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297: No bootstrap required, opened a new log
I20260806 00:54:15.979478 5512 ts_tablet_manager.cc:1397] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:54:15.980352 5513 tablet_bootstrap.cc:492] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: No bootstrap required, opened a new log
I20260806 00:54:15.980499 5513 ts_tablet_manager.cc:1397] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 00:54:15.983441 5512 raft_consensus.cc:348] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:15.983754 5512 raft_consensus.cc:374] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:15.983837 5512 raft_consensus.cc:729] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3096c6aa5f340f58b7d3b0a68711297, State: Initialized, Role: FOLLOWER
I20260806 00:54:15.983989 5513 raft_consensus.cc:348] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:15.984303 5513 raft_consensus.cc:374] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:15.984382 5513 raft_consensus.cc:729] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda, State: Initialized, Role: FOLLOWER
I20260806 00:54:15.984355 5512 consensus_queue.cc:260] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:15.984827 5513 consensus_queue.cc:260] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:15.985488 5512 ts_tablet_manager.cc:1428] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:54:15.985729 5513 ts_tablet_manager.cc:1428] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 00:54:15.986922 5084 heartbeater.cc:503] Master 127.27.77.254:38937 was elected leader, sending a full tablet report...
I20260806 00:54:15.986963 5287 heartbeater.cc:503] Master 127.27.77.254:38937 was elected leader, sending a full tablet report...
I20260806 00:54:15.987185 5514 tablet_bootstrap.cc:492] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290: No bootstrap required, opened a new log
I20260806 00:54:15.987390 5514 ts_tablet_manager.cc:1397] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260806 00:54:15.993319 5514 raft_consensus.cc:348] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:15.993628 5514 raft_consensus.cc:374] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:15.993705 5514 raft_consensus.cc:729] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb3523d7e6f84f349d3bb5c864757290, State: Initialized, Role: FOLLOWER
I20260806 00:54:15.994177 5514 consensus_queue.cc:260] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:15.994877 5469 heartbeater.cc:503] Master 127.27.77.254:38937 was elected leader, sending a full tablet report...
I20260806 00:54:15.995265 5514 ts_tablet_manager.cc:1428] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
W20260806 00:54:16.150100 5474 tablet.cc:2369] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:54:16.164263 5291 tablet.cc:2369] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:54:16.181325 5086 tablet.cc:2369] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:54:16.376124 5521 raft_consensus.cc:482] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:54:16.376346 5521 raft_consensus.cc:504] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:16.377524 5521 leader_election.cc:288] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb3523d7e6f84f349d3bb5c864757290 (127.27.77.195:42489), 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda (127.27.77.193:39337)
I20260806 00:54:16.383291 5510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:16.377966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:39373 (local address 127.27.77.193:39337)
0806 00:54:16.378185 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:16.378190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:16.378348 (+ 158us) server_negotiation.cc:407] Connection header received
0806 00:54:16.378454 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:16.378457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:16.378520 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:16.378629 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:16.379018 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.379857 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.381889 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.382528 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.382698 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:16.382754 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:16.383068 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 00:54:16.383159 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260806 00:54:16.383291 5506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:16.378009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:59525 (local address 127.27.77.195:42489)
0806 00:54:16.378188 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:16.378193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:16.378412 (+ 219us) server_negotiation.cc:407] Connection header received
0806 00:54:16.378472 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:16.378475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:16.378520 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:16.378612 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:16.379018 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.379857 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.381889 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.382573 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.382698 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:16.382754 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:16.383068 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 00:54:16.383159 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 00:54:16.383287 5525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:16.377916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:42489 (local address 127.27.77.194:59525)
0806 00:54:16.378299 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:16.378311 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:16.378400 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:16.378644 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:16.378647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:16.378685 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:16.378887 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:16.378893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.379985 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.379988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:16.381737 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:16.381751 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.382733 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.382738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:16.382851 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:16.382922 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 00:54:16.383003 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 00:54:16.383097 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":343,"spinlock_wait_cycles":17280,"thread_start_us":134,"threads_started":1}
I20260806 00:54:16.383287 5524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:16.377823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:39337 (local address 127.27.77.194:39373)
0806 00:54:16.378166 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:16.378183 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:16.378319 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:16.378644 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:16.378647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:16.378684 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:16.378886 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:16.378893 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.379985 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.379988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:16.381737 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:16.381751 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.382700 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.382704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:16.382825 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:16.382917 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 00:54:16.383003 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 00:54:16.383130 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":263,"spinlock_wait_cycles":6272,"thread_start_us":99,"threads_started":1}
I20260806 00:54:16.384332 5409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be920ffcaae49da80a09a24338f3863" candidate_uuid: "e3096c6aa5f340f58b7d3b0a68711297" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb3523d7e6f84f349d3bb5c864757290" is_pre_election: true
I20260806 00:54:16.384339 5018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be920ffcaae49da80a09a24338f3863" candidate_uuid: "e3096c6aa5f340f58b7d3b0a68711297" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" is_pre_election: true
I20260806 00:54:16.384677 5409 raft_consensus.cc:2454] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3096c6aa5f340f58b7d3b0a68711297 in term 0.
I20260806 00:54:16.384735 5018 raft_consensus.cc:2454] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3096c6aa5f340f58b7d3b0a68711297 in term 0.
I20260806 00:54:16.385315 5123 leader_election.cc:302] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda, cb3523d7e6f84f349d3bb5c864757290, e3096c6aa5f340f58b7d3b0a68711297; no voters:
I20260806 00:54:16.385687 5521 raft_consensus.cc:2781] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:54:16.385784 5521 raft_consensus.cc:482] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:54:16.385869 5521 raft_consensus.cc:3037] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:16.387127 5521 raft_consensus.cc:504] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:16.387609 5521 leader_election.cc:288] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [CANDIDATE]: Term 1 election: Requested vote from peers cb3523d7e6f84f349d3bb5c864757290 (127.27.77.195:42489), 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda (127.27.77.193:39337)
I20260806 00:54:16.388069 5018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be920ffcaae49da80a09a24338f3863" candidate_uuid: "e3096c6aa5f340f58b7d3b0a68711297" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda"
I20260806 00:54:16.388203 5018 raft_consensus.cc:3037] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:16.389587 5018 raft_consensus.cc:2454] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3096c6aa5f340f58b7d3b0a68711297 in term 1.
I20260806 00:54:16.389945 5123 leader_election.cc:302] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda, e3096c6aa5f340f58b7d3b0a68711297; no voters:
I20260806 00:54:16.389919 5409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be920ffcaae49da80a09a24338f3863" candidate_uuid: "e3096c6aa5f340f58b7d3b0a68711297" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb3523d7e6f84f349d3bb5c864757290"
I20260806 00:54:16.390086 5409 raft_consensus.cc:3037] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:16.390162 5521 raft_consensus.cc:2781] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:16.390380 5521 raft_consensus.cc:686] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [term 1 LEADER]: Becoming Leader. State: Replica: e3096c6aa5f340f58b7d3b0a68711297, State: Running, Role: LEADER
I20260806 00:54:16.390810 5521 consensus_queue.cc:237] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:16.391682 5409 raft_consensus.cc:2454] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3096c6aa5f340f58b7d3b0a68711297 in term 1.
I20260806 00:54:16.395375 3779 catalog_manager.cc:5495] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3096c6aa5f340f58b7d3b0a68711297 (127.27.77.194). New cstate: current_term: 1 leader_uuid: "e3096c6aa5f340f58b7d3b0a68711297" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } health_report { overall_health: UNKNOWN } } }
I20260806 00:54:16.415293 5511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:16.411559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36850 (local address 127.27.77.194:44163)
0806 00:54:16.411710 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:16.411715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:16.411732 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:54:16.411829 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:16.411832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:16.411888 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:16.412001 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:54:16.412258 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.412839 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.413727 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.413949 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.414058 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:16.414378 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:54:16.414892 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:54:16.415047 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:16.415083 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 00:54:16.415141 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:54:16.421202 5018 raft_consensus.cc:1261] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 1 FOLLOWER]: Refusing update from remote peer e3096c6aa5f340f58b7d3b0a68711297: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:16.421260 5409 raft_consensus.cc:1261] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290 [term 1 FOLLOWER]: Refusing update from remote peer e3096c6aa5f340f58b7d3b0a68711297: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:16.421977 5521 consensus_queue.cc:1035] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:16.422206 5526 consensus_queue.cc:1035] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:16.433030 5534 mvcc.cc:205] Tried to move back new op lower bound from 7315364480693653504 to 7315364480580734976. Current Snapshot: MvccSnapshot[applied={T|T < 7315364480693653504}]
I20260806 00:54:16.434798 5532 mvcc.cc:205] Tried to move back new op lower bound from 7315364480693653504 to 7315364480580734976. Current Snapshot: MvccSnapshot[applied={T|T < 7315364480693653504}]
I20260806 00:54:16.447399 3780 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:16.447620 3780 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:16.452464 3780 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9a22350b1e44dcc935e18a905203e01: Sending DeleteTablet for 3 replicas of tablet 4be920ffcaae49da80a09a24338f3863
I20260806 00:54:16.453506 4991 tablet_service.cc:1514] Processing DeleteTablet for tablet 4be920ffcaae49da80a09a24338f3863 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:16 UTC) from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:60742
I20260806 00:54:16.453581 5189 tablet_service.cc:1514] Processing DeleteTablet for tablet 4be920ffcaae49da80a09a24338f3863 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:16 UTC) from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:36848
I20260806 00:54:16.454097 5382 tablet_service.cc:1514] Processing DeleteTablet for tablet 4be920ffcaae49da80a09a24338f3863 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:16 UTC) from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:42734
I20260806 00:54:16.454963 5546 tablet_replica.cc:330] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297: stopping tablet replica
I20260806 00:54:16.455340 5546 raft_consensus.cc:2229] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:54:16.455680 5552 tablet_replica.cc:330] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290: stopping tablet replica
I20260806 00:54:16.455875 5546 raft_consensus.cc:2258] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:16.455937 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 3738
I20260806 00:54:16.456079 5547 tablet_replica.cc:330] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: stopping tablet replica
I20260806 00:54:16.456013 5552 raft_consensus.cc:2229] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:16.456802 5552 raft_consensus.cc:2258] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:16.456791 5547 raft_consensus.cc:2229] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:16.457165 5547 raft_consensus.cc:2258] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:16.459777 5547 ts_tablet_manager.cc:1905] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:16.459950 5546 ts_tablet_manager.cc:1905] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:16.462944 5547 ts_tablet_manager.cc:1918] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:16.463129 5547 log.cc:1196] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-0/wal/wals/4be920ffcaae49da80a09a24338f3863
I20260806 00:54:16.464059 5547 ts_tablet_manager.cc:1939] T 4be920ffcaae49da80a09a24338f3863 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: Deleting consensus metadata
I20260806 00:54:16.468351 5546 ts_tablet_manager.cc:1918] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:16.468498 5546 log.cc:1196] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/4be920ffcaae49da80a09a24338f3863
I20260806 00:54:16.468883 5546 ts_tablet_manager.cc:1939] T 4be920ffcaae49da80a09a24338f3863 P e3096c6aa5f340f58b7d3b0a68711297: Deleting consensus metadata
W20260806 00:54:16.469998 4937 connection.cc:535] server connection from 127.0.0.1:60742 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 00:54:16.470551 5123 connection.cc:441] server connection from 127.0.0.1:36848 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36848 (request call id 1) could send its response
I20260806 00:54:16.465425 5552 ts_tablet_manager.cc:1905] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:16.471470 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:16.454421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53570 (local address 127.27.77.253:45881)
0806 00:54:16.457398 (+ 2977us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:16.457405 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:16.457421 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:54:16.457463 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:16.457468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:16.457521 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:16.457618 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:16.461414 (+ 3796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.462274 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.463859 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.464321 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.464440 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:16.466274 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:16.466278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:16.466281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:16.466310 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:16.468282 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:16.469090 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:16.469096 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:16.469101 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:16.469201 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:16.469528 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:16.469531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:16.469533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:16.469799 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:16.469923 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:16.471125 (+ 1202us) server_negotiation.cc:299] Negotiation successful
0806 00:54:16.471280 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2888,"thread_start_us":68,"threads_started":1}
I20260806 00:54:16.474887 5552 ts_tablet_manager.cc:1918] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:16.475049 5552 log.cc:1196] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/4be920ffcaae49da80a09a24338f3863
I20260806 00:54:16.475505 5552 ts_tablet_manager.cc:1939] T 4be920ffcaae49da80a09a24338f3863 P cb3523d7e6f84f349d3bb5c864757290: Deleting consensus metadata
W20260806 00:54:16.477002 5316 connection.cc:441] server connection from 127.0.0.1:42734 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42734 (request call id 1) could send its response
I20260806 00:54:16.480409 5554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:16.455208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44368 (local address 127.27.77.252:43643)
0806 00:54:16.455918 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:16.455923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:16.470955 (+ 15032us) server_negotiation.cc:407] Connection header received
0806 00:54:16.471056 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:16.471066 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:16.471130 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:16.471222 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:16.472188 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.473021 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.474141 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.474476 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.474553 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:16.476456 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:16.476461 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:16.476464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:16.476491 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:16.478466 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:16.478952 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:16.478955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:16.478957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:16.479057 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:16.479386 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:16.479389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:16.479391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:16.479708 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:16.479837 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:16.480076 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 00:54:16.480206 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":88,"threads_started":1}
I20260806 00:54:16.486238 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:16.475569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53584 (local address 127.27.77.253:45881)
0806 00:54:16.475716 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:16.475720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:16.476795 (+ 1075us) server_negotiation.cc:407] Connection header received
0806 00:54:16.476996 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:16.477001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:16.477054 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:16.477113 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:16.478097 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.478980 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.480428 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.480775 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.480909 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:16.482355 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:16.482359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:16.482362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:16.482392 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:16.484400 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:16.484987 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:16.484991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:16.484993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:16.485082 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:16.485363 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:16.485367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:16.485369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:16.485545 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:16.485691 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:16.485922 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 00:54:16.486050 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:54:16.488111 5563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:16.475167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44376 (local address 127.27.77.252:43643)
0806 00:54:16.475967 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:16.475972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:16.476439 (+ 467us) server_negotiation.cc:407] Connection header received
0806 00:54:16.476604 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:16.476609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:16.476657 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:16.476749 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:16.478225 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.478987 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.482134 (+ 3147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:16.482481 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:16.482580 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:16.484170 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:16.484173 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:16.484176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:16.484207 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:16.486007 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:16.486546 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:16.486550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:16.486551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:16.486674 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:16.487011 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:16.487015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:16.487017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:16.487242 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:16.487378 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:16.487666 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 00:54:16.487930 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":83,"threads_started":1}
I20260806 00:54:17.438388 5567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:17.437817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.195:42893)
0806 00:54:17.438160 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:17.438258 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":78,"threads_started":1}
I20260806 00:54:17.438736 5566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:17.437781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.193:50253)
0806 00:54:17.438354 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:17.438455 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":77,"threads_started":1}
W20260806 00:54:17.438814 5469 heartbeater.cc:644] Failed to heartbeat to 127.27.77.254:38937 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
I20260806 00:54:17.438876 5568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:17.437862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.194:60997)
0806 00:54:17.438686 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:17.438772 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":80,"threads_started":1}
W20260806 00:54:17.439023 5084 heartbeater.cc:644] Failed to heartbeat to 127.27.77.254:38937 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
W20260806 00:54:17.439329 5287 heartbeater.cc:644] Failed to heartbeat to 127.27.77.254:38937 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
I20260806 00:54:17.952612 5569 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a22350b1e44dcc935e18a905203e01)
I20260806 00:54:17.952873 5569 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } }
I20260806 00:54:17.953948 5569 leader_election.cc:288] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a22350b1e44dcc935e18a905203e01 (127.27.77.254:38937), e239ce00299a4509830746a4126c44cf (127.27.77.252:43643)
I20260806 00:54:17.954630 5571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:17.954146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.0.0.1:43474)
0806 00:54:17.954463 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:17.954535 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":45,"thread_start_us":75,"threads_started":1}
W20260806 00:54:17.954881 4571 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
I20260806 00:54:17.955837 5571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:17.955578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.0.0.1:43486)
0806 00:54:17.955706 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:17.955766 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 00:54:17.955858 5574 raft_consensus.cc:482] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a22350b1e44dcc935e18a905203e01)
W20260806 00:54:17.956007 4571 leader_election.cc:334] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e9a22350b1e44dcc935e18a905203e01 (127.27.77.254:38937): Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
I20260806 00:54:17.956043 5574 raft_consensus.cc:504] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } }
I20260806 00:54:17.956950 5574 leader_election.cc:288] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a22350b1e44dcc935e18a905203e01 (127.27.77.254:38937), bbdf6367d78047ddafb01c62630ff258 (127.27.77.253:45881)
I20260806 00:54:17.957542 5576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:17.957021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.0.0.1:43494)
0806 00:54:17.957348 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:17.957446 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":90,"threads_started":1}
W20260806 00:54:17.957715 4827 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
I20260806 00:54:17.958618 5570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:17.954146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:43643 (local address 127.0.0.1:44384)
0806 00:54:17.954472 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:17.954489 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:17.954602 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:17.954938 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:17.954941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:17.954960 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:17.955098 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:17.955105 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:17.956217 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:17.956222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:17.957186 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:17.957199 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:17.958122 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:17.958125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:17.958263 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:17.958284 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:54:17.958381 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 00:54:17.958474 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":32,"thread_start_us":71,"threads_started":1}
I20260806 00:54:17.958736 5576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:17.958471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.0.0.1:43510)
0806 00:54:17.958594 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:17.958674 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 00:54:17.958843 4827 leader_election.cc:334] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e9a22350b1e44dcc935e18a905203e01 (127.27.77.254:38937): Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
I20260806 00:54:17.959036 5572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:17.954383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44384 (local address 127.27.77.252:43643)
0806 00:54:17.954637 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:17.954644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:17.954661 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:54:17.954749 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:17.954752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:17.954803 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:17.954895 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:17.955233 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:17.956075 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:17.957337 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:17.957992 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:17.958112 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:17.958176 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:17.958860 (+ 684us) server_negotiation.cc:299] Negotiation successful
0806 00:54:17.958940 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":81,"threads_started":1}
I20260806 00:54:17.959491 4859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bbdf6367d78047ddafb01c62630ff258" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e239ce00299a4509830746a4126c44cf" is_pre_election: true
I20260806 00:54:17.959736 4859 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbdf6367d78047ddafb01c62630ff258 in term 1.
I20260806 00:54:17.960115 4573 leader_election.cc:302] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bbdf6367d78047ddafb01c62630ff258, e239ce00299a4509830746a4126c44cf; no voters: e9a22350b1e44dcc935e18a905203e01
I20260806 00:54:17.960480 5569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 00:54:17.960556 5569 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a22350b1e44dcc935e18a905203e01)
I20260806 00:54:17.960601 5569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:54:17.961503 5578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:17.957252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53586 (local address 127.27.77.253:45881)
0806 00:54:17.957534 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:17.957543 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:17.957561 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:54:17.957684 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:17.957687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:17.957728 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:17.957806 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:17.958186 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:17.959075 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:17.960395 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:17.960944 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:17.961063 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:17.961121 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:17.961317 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 00:54:17.961389 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":224,"thread_start_us":111,"threads_started":1}
I20260806 00:54:17.961513 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:17.957172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:45881 (local address 127.0.0.1:53586)
0806 00:54:17.957462 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:17.957475 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:17.957559 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:17.957883 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:17.957887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:17.957910 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:17.958033 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:17.958041 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:17.959176 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:17.959181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:17.960265 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:17.960275 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:17.961076 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:17.961080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:17.961181 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:17.961203 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:54:17.961299 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 00:54:17.961370 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":74,"threads_started":1}
I20260806 00:54:17.961809 5569 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } }
I20260806 00:54:17.961951 4605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e239ce00299a4509830746a4126c44cf" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bbdf6367d78047ddafb01c62630ff258" is_pre_election: true
I20260806 00:54:17.962203 4605 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e239ce00299a4509830746a4126c44cf in current term 2: Already voted for candidate bbdf6367d78047ddafb01c62630ff258 in this term.
I20260806 00:54:17.962275 5569 leader_election.cc:288] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [CANDIDATE]: Term 2 election: Requested vote from peers e9a22350b1e44dcc935e18a905203e01 (127.27.77.254:38937), e239ce00299a4509830746a4126c44cf (127.27.77.252:43643)
I20260806 00:54:17.962590 4859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bbdf6367d78047ddafb01c62630ff258" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e239ce00299a4509830746a4126c44cf"
I20260806 00:54:17.962684 5570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:17.962413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.0.0.1:43512)
0806 00:54:17.962547 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:17.962612 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 00:54:17.962690 4859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:54:17.963279 5570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:17.963095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.0.0.1:43520)
0806 00:54:17.963180 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:17.963224 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 00:54:17.963383 4571 leader_election.cc:334] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e9a22350b1e44dcc935e18a905203e01 (127.27.77.254:38937): Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
I20260806 00:54:17.963845 4859 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbdf6367d78047ddafb01c62630ff258 in term 2.
I20260806 00:54:17.964233 4573 leader_election.cc:302] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bbdf6367d78047ddafb01c62630ff258, e239ce00299a4509830746a4126c44cf; no voters: e9a22350b1e44dcc935e18a905203e01
I20260806 00:54:17.964272 4831 leader_election.cc:302] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e239ce00299a4509830746a4126c44cf; no voters: bbdf6367d78047ddafb01c62630ff258, e9a22350b1e44dcc935e18a905203e01
I20260806 00:54:17.964419 5569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 00:54:17.964499 5574 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:54:17.964644 5569 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [term 2 LEADER]: Becoming Leader. State: Replica: bbdf6367d78047ddafb01c62630ff258, State: Running, Role: LEADER
I20260806 00:54:17.964915 5569 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } }
I20260806 00:54:17.965929 5580 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbdf6367d78047ddafb01c62630ff258. Latest consensus state: current_term: 2 leader_uuid: "bbdf6367d78047ddafb01c62630ff258" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } } }
I20260806 00:54:17.966058 5580 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:17.966370 5582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:17.968720 5582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:17.969121 5582 catalog_manager.cc:1259] Loaded cluster ID: 382f6c59a82144088da901f2ccc4a5ad
I20260806 00:54:17.969170 5582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:17.969715 5582 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:17.970106 5582 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: Loaded TSK: 0
I20260806 00:54:17.970775 5582 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:54:18.008216 4594 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:54:18.011691 4594 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:54:18.015276 5570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.014885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.0.0.1:43542)
0806 00:54:18.015117 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.015187 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 00:54:18.015328 4859 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [term 2 FOLLOWER]: Refusing update from remote peer bbdf6367d78047ddafb01c62630ff258: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 00:54:18.015960 5569 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 00:54:18.016225 5570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.015929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.0.0.1:43548)
0806 00:54:18.016047 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.016139 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 00:54:18.016342 4571 consensus_peers.cc:487] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 -> Peer e9a22350b1e44dcc935e18a905203e01 (127.27.77.254:38937): Couldn't send request to peer e9a22350b1e44dcc935e18a905203e01. Status: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 00:54:18.017740 5574 sys_catalog.cc:455] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbdf6367d78047ddafb01c62630ff258. Latest consensus state: current_term: 2 leader_uuid: "bbdf6367d78047ddafb01c62630ff258" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } } }
I20260806 00:54:18.017886 5574 sys_catalog.cc:458] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:54:18.019053 5580 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "bbdf6367d78047ddafb01c62630ff258" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } } }
I20260806 00:54:18.019202 5580 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:18.019423 5574 sys_catalog.cc:455] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "bbdf6367d78047ddafb01c62630ff258" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e9a22350b1e44dcc935e18a905203e01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38937 } } peers { permanent_uuid: "bbdf6367d78047ddafb01c62630ff258" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 45881 } attrs { promote: false } } peers { permanent_uuid: "e239ce00299a4509830746a4126c44cf" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 43643 } attrs { promote: false } } }
I20260806 00:54:18.019593 5574 sys_catalog.cc:458] T 00000000000000000000000000000000 P e239ce00299a4509830746a4126c44cf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:54:18.031195 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.026552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36856 (local address 127.27.77.194:44163)
0806 00:54:18.027179 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:18.027194 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:18.027212 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:54:18.027271 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:18.027274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:18.027395 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:18.027856 (+ 461us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:18.027969 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.028782 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.030074 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.030543 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.030665 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:18.030736 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:18.030957 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 00:54:18.031048 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":109,"threads_started":1}
I20260806 00:54:18.031194 5570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.026427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:44163 (local address 127.0.0.1:36856)
0806 00:54:18.026650 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.026668 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:18.026750 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:18.027603 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:18.027607 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:18.027626 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:18.027738 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:18.027745 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.028907 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.028910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:18.029928 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:18.029939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.030674 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.030677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:18.030815 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:18.030841 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:54:18.030937 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 00:54:18.031016 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":101}
I20260806 00:54:18.031808 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.026990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:39337 (local address 127.0.0.1:60748)
0806 00:54:18.027390 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.027407 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:18.027502 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:18.028143 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:18.028146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:18.028164 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:18.028285 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:18.028290 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.029154 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.029157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:18.030342 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:18.030353 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.031365 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.031369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:18.031455 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:18.031480 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 00:54:18.031564 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 00:54:18.031632 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":49,"thread_start_us":99,"threads_started":1}
I20260806 00:54:18.032161 5189 tablet_service.cc:1467] Processing CreateTablet for tablet 1649cbd8a28048b5a1019229e26bab59 (DEFAULT_TABLE table=test-table [id=c8f66b561bf14b24a22e0cb1abe1de68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:18.032439 5592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.027615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60748 (local address 127.27.77.193:39337)
0806 00:54:18.027868 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:18.027871 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:18.027885 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:54:18.027977 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:18.027980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:18.028032 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:18.028135 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:18.028415 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.029021 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.030486 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.031236 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.032075 (+ 839us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:18.032148 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:18.032187 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 00:54:18.032275 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":90,"threads_started":1}
I20260806 00:54:18.032495 5189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1649cbd8a28048b5a1019229e26bab59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:18.032552 5571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.026737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:42489 (local address 127.0.0.1:42748)
0806 00:54:18.026837 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.026853 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:18.026945 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:18.027877 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:18.027880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:18.027897 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:18.027975 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:18.027980 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.029109 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.029114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:18.031099 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:18.031110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.032115 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.032119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:18.032235 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:18.032261 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:54:18.032344 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 00:54:18.032435 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":795}
I20260806 00:54:18.032617 5591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.027332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42748 (local address 127.27.77.195:42489)
0806 00:54:18.027632 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:18.027638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:18.027653 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:54:18.027704 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:18.027707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:18.027753 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:18.028079 (+ 326us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:18.028194 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.028984 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.031286 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.031988 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.032105 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:18.032182 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:18.032389 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 00:54:18.032488 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":80,"threads_started":1}
I20260806 00:54:18.033135 5382 tablet_service.cc:1467] Processing CreateTablet for tablet 1649cbd8a28048b5a1019229e26bab59 (DEFAULT_TABLE table=test-table [id=c8f66b561bf14b24a22e0cb1abe1de68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:18.033226 4991 tablet_service.cc:1467] Processing CreateTablet for tablet 1649cbd8a28048b5a1019229e26bab59 (DEFAULT_TABLE table=test-table [id=c8f66b561bf14b24a22e0cb1abe1de68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:18.033362 5382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1649cbd8a28048b5a1019229e26bab59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:18.033475 4991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1649cbd8a28048b5a1019229e26bab59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:18.035364 5594 tablet_bootstrap.cc:492] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297: Bootstrap starting.
I20260806 00:54:18.036278 5596 tablet_bootstrap.cc:492] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290: Bootstrap starting.
I20260806 00:54:18.036309 5599 tablet_bootstrap.cc:492] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: Bootstrap starting.
I20260806 00:54:18.036620 5594 tablet_bootstrap.cc:654] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:18.037173 5596 tablet_bootstrap.cc:654] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:18.037302 5599 tablet_bootstrap.cc:654] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:18.037984 5594 tablet_bootstrap.cc:492] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297: No bootstrap required, opened a new log
I20260806 00:54:18.038096 5594 ts_tablet_manager.cc:1397] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:54:18.038635 5594 raft_consensus.cc:348] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:18.038728 5596 tablet_bootstrap.cc:492] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290: No bootstrap required, opened a new log
I20260806 00:54:18.038765 5594 raft_consensus.cc:374] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:18.038800 5594 raft_consensus.cc:729] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3096c6aa5f340f58b7d3b0a68711297, State: Initialized, Role: FOLLOWER
I20260806 00:54:18.038810 5596 ts_tablet_manager.cc:1397] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:54:18.039143 5594 consensus_queue.cc:260] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:18.039278 5596 raft_consensus.cc:348] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:18.039402 5596 raft_consensus.cc:374] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:18.039444 5596 raft_consensus.cc:729] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb3523d7e6f84f349d3bb5c864757290, State: Initialized, Role: FOLLOWER
I20260806 00:54:18.039583 5594 ts_tablet_manager.cc:1428] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 00:54:18.039618 5596 consensus_queue.cc:260] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:18.039985 5596 ts_tablet_manager.cc:1428] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 00:54:18.040868 5599 tablet_bootstrap.cc:492] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: No bootstrap required, opened a new log
I20260806 00:54:18.040961 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.040033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.194:51821)
0806 00:54:18.040764 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.040828 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":105,"threads_started":1}
I20260806 00:54:18.041008 5599 ts_tablet_manager.cc:1397] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 00:54:18.041390 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.040516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.195:38145)
0806 00:54:18.041218 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.041279 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":98,"threads_started":1}
I20260806 00:54:18.041468 5599 raft_consensus.cc:348] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:18.041618 5599 raft_consensus.cc:374] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:18.041666 5599 raft_consensus.cc:729] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda, State: Initialized, Role: FOLLOWER
I20260806 00:54:18.041807 5599 consensus_queue.cc:260] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:18.042104 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.041872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.195:51571)
0806 00:54:18.041986 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.042038 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 00:54:18.042160 5599 ts_tablet_manager.cc:1428] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:54:18.043033 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.042621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.193:34015)
0806 00:54:18.042868 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.042927 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":92,"threads_started":1}
I20260806 00:54:18.045972 5473 heartbeater.cc:503] Master 127.27.77.253:45881 was elected leader, sending a full tablet report...
I20260806 00:54:18.047363 5288 heartbeater.cc:503] Master 127.27.77.253:45881 was elected leader, sending a full tablet report...
I20260806 00:54:18.047932 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.047642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.194:56429)
0806 00:54:18.047794 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.047857 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 00:54:18.267774 5610 raft_consensus.cc:482] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:54:18.267961 5610 raft_consensus.cc:504] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:18.269207 5610 leader_election.cc:288] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3096c6aa5f340f58b7d3b0a68711297 (127.27.77.194:44163), cb3523d7e6f84f349d3bb5c864757290 (127.27.77.195:42489)
I20260806 00:54:18.273471 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.269572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:58153 (local address 127.27.77.194:44163)
0806 00:54:18.269706 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:18.269710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:18.269988 (+ 278us) server_negotiation.cc:407] Connection header received
0806 00:54:18.270164 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:18.270167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:18.270225 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:18.270285 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:18.270629 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.271221 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.272417 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.272851 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.272985 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:18.273047 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:18.273266 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 00:54:18.273347 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 00:54:18.273474 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.269462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:44163 (local address 127.27.77.193:58153)
0806 00:54:18.269882 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.269894 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:18.270031 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:18.270326 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:18.270329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:18.270345 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:18.270499 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:18.270505 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.271351 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.271354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:18.272312 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:18.272322 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.272980 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.272984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:18.273094 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:18.273118 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 00:54:18.273243 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 00:54:18.273319 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":83,"spinlock_wait_cycles":7936}
I20260806 00:54:18.273638 5591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.269607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:46679 (local address 127.27.77.195:42489)
0806 00:54:18.269748 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:18.269753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:18.270013 (+ 260us) server_negotiation.cc:407] Connection header received
0806 00:54:18.270191 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:18.270193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:18.270237 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:18.270330 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:18.270654 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.271156 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.272506 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.273028 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.273197 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:18.273251 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:18.273433 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 00:54:18.273507 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:54:18.273746 5613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.269482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:42489 (local address 127.27.77.193:46679)
0806 00:54:18.269882 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.269894 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:18.270042 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:18.270364 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:18.270366 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:18.270393 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:18.270512 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:18.270517 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.271276 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.271280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:18.272351 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:18.272363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.273198 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.273201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:18.273301 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:18.273327 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:54:18.273417 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 00:54:18.273490 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":103,"spinlock_wait_cycles":12160,"thread_start_us":184,"threads_started":1}
I20260806 00:54:18.274479 5220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1649cbd8a28048b5a1019229e26bab59" candidate_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3096c6aa5f340f58b7d3b0a68711297" is_pre_election: true
I20260806 00:54:18.274528 5409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1649cbd8a28048b5a1019229e26bab59" candidate_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb3523d7e6f84f349d3bb5c864757290" is_pre_election: true
I20260806 00:54:18.274696 5409 raft_consensus.cc:2454] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda in term 0.
I20260806 00:54:18.274749 5220 raft_consensus.cc:2454] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda in term 0.
I20260806 00:54:18.275097 4935 leader_election.cc:302] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda, cb3523d7e6f84f349d3bb5c864757290; no voters:
I20260806 00:54:18.275373 5610 raft_consensus.cc:2781] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:54:18.275468 5610 raft_consensus.cc:482] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:54:18.275563 5610 raft_consensus.cc:3037] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:18.276633 5610 raft_consensus.cc:504] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:18.277016 5610 leader_election.cc:288] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [CANDIDATE]: Term 1 election: Requested vote from peers e3096c6aa5f340f58b7d3b0a68711297 (127.27.77.194:44163), cb3523d7e6f84f349d3bb5c864757290 (127.27.77.195:42489)
I20260806 00:54:18.277367 5409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1649cbd8a28048b5a1019229e26bab59" candidate_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb3523d7e6f84f349d3bb5c864757290"
I20260806 00:54:18.277439 5220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1649cbd8a28048b5a1019229e26bab59" candidate_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3096c6aa5f340f58b7d3b0a68711297"
I20260806 00:54:18.277498 5409 raft_consensus.cc:3037] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:18.277583 5220 raft_consensus.cc:3037] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:18.278432 5409 raft_consensus.cc:2454] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda in term 1.
I20260806 00:54:18.278616 5220 raft_consensus.cc:2454] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda in term 1.
I20260806 00:54:18.278720 4935 leader_election.cc:302] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda, cb3523d7e6f84f349d3bb5c864757290; no voters:
I20260806 00:54:18.278908 5610 raft_consensus.cc:2781] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:18.279155 5610 raft_consensus.cc:686] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 1 LEADER]: Becoming Leader. State: Replica: 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda, State: Running, Role: LEADER
I20260806 00:54:18.279407 5085 heartbeater.cc:503] Master 127.27.77.253:45881 was elected leader, sending a full tablet report...
I20260806 00:54:18.279412 5610 consensus_queue.cc:237] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } } peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } }
I20260806 00:54:18.279906 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.279658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.193:54691)
0806 00:54:18.279770 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.279835 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 00:54:18.282122 4592 catalog_manager.cc:5495] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda (127.27.77.193). New cstate: current_term: 1 leader_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f1e8a0fcb7d45d7a0e6e3d4cabeeeda" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 39337 } health_report { overall_health: HEALTHY } } }
I20260806 00:54:18.330817 5592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.327252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60752 (local address 127.27.77.193:39337)
0806 00:54:18.327385 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:18.327389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:18.327404 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:54:18.327567 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:18.327570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:18.327646 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:18.327736 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:54:18.327959 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.328486 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.329312 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:18.329518 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:18.329607 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:18.329816 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:54:18.330273 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:54:18.330378 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:18.330528 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 00:54:18.330578 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:54:18.333966 5409 raft_consensus.cc:1261] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290 [term 1 FOLLOWER]: Refusing update from remote peer 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:18.334270 5220 raft_consensus.cc:1261] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297 [term 1 FOLLOWER]: Refusing update from remote peer 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:18.334566 5610 consensus_queue.cc:1035] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb3523d7e6f84f349d3bb5c864757290" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:18.334900 5614 consensus_queue.cc:1035] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3096c6aa5f340f58b7d3b0a68711297" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:18.336354 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.336007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.195:41103)
0806 00:54:18.336178 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.336265 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260806 00:54:18.338892 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.337702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.194:60933)
0806 00:54:18.338753 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.338815 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":973}
I20260806 00:54:18.339572 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.338977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.193:59187)
0806 00:54:18.339412 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.339470 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361}
I20260806 00:54:18.339872 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.339224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.195:47487)
0806 00:54:18.339739 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.339796 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449}
I20260806 00:54:18.342921 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.342670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.193:40655)
0806 00:54:18.342790 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.342848 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 00:54:18.343174 5622 mvcc.cc:205] Tried to move back new op lower bound from 7315364488530083840 to 7315364488316076032. Current Snapshot: MvccSnapshot[applied={T|T < 7315364488530083840}]
I20260806 00:54:18.348738 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:18.348283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38937 (local address 127.27.77.194:36357)
0806 00:54:18.348594 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.348657 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225}
I20260806 00:54:18.348734 4592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:18.348901 4592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:18.353791 4592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bbdf6367d78047ddafb01c62630ff258: Sending DeleteTablet for 3 replicas of tablet 1649cbd8a28048b5a1019229e26bab59
I20260806 00:54:18.354619 5189 tablet_service.cc:1514] Processing DeleteTablet for tablet 1649cbd8a28048b5a1019229e26bab59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:18 UTC) from {username='oryx', principal='oryx/127.27.77.253@KRBTEST.COM'} at 127.0.0.1:36856
I20260806 00:54:18.354643 4991 tablet_service.cc:1514] Processing DeleteTablet for tablet 1649cbd8a28048b5a1019229e26bab59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:18 UTC) from {username='oryx', principal='oryx/127.27.77.253@KRBTEST.COM'} at 127.0.0.1:60748
I20260806 00:54:18.354682 5185 tablet_service.cc:1514] Processing DeleteTablet for tablet 1649cbd8a28048b5a1019229e26bab59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:18 UTC) from {username='oryx', principal='oryx/127.27.77.253@KRBTEST.COM'} at 127.0.0.1:36856
I20260806 00:54:18.355309 5382 tablet_service.cc:1514] Processing DeleteTablet for tablet 1649cbd8a28048b5a1019229e26bab59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:18 UTC) from {username='oryx', principal='oryx/127.27.77.253@KRBTEST.COM'} at 127.0.0.1:42748
I20260806 00:54:18.355378 5635 tablet_replica.cc:330] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda: stopping tablet replica
I20260806 00:54:18.355544 5635 raft_consensus.cc:2229] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:54:18.355881 5635 raft_consensus.cc:2258] T 1649cbd8a28048b5a1019229e26bab59 P 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:18.355963 5636 tablet_replica.cc:330] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290: stopping tablet replica
I20260806 00:54:18.356102 5636 raft_consensus.cc:2229] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:18.356185 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 4786
I20260806 00:54:18.356240 5636 raft_consensus.cc:2258] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:18.356879 5634 tablet_replica.cc:330] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297: stopping tablet replica
I20260806 00:54:18.357016 5634 raft_consensus.cc:2229] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:18.357177 5634 raft_consensus.cc:2258] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:18.357312 5636 ts_tablet_manager.cc:1905] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:18.360423 5636 ts_tablet_manager.cc:1918] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:18.360520 5636 log.cc:1196] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/1649cbd8a28048b5a1019229e26bab59
I20260806 00:54:18.360893 5636 ts_tablet_manager.cc:1939] T 1649cbd8a28048b5a1019229e26bab59 P cb3523d7e6f84f349d3bb5c864757290: Deleting consensus metadata
I20260806 00:54:18.362877 5634 ts_tablet_manager.cc:1905] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:18.365769 4571 catalog_manager.cc:4841] TS cb3523d7e6f84f349d3bb5c864757290 (127.27.77.195:42489): tablet 1649cbd8a28048b5a1019229e26bab59 (table test-table [id=c8f66b561bf14b24a22e0cb1abe1de68]) successfully deleted
I20260806 00:54:18.365988 5634 ts_tablet_manager.cc:1918] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:18.366088 5634 log.cc:1196] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/1649cbd8a28048b5a1019229e26bab59
I20260806 00:54:18.366333 5634 ts_tablet_manager.cc:1939] T 1649cbd8a28048b5a1019229e26bab59 P e3096c6aa5f340f58b7d3b0a68711297: Deleting consensus metadata
I20260806 00:54:18.367105 4575 catalog_manager.cc:4841] TS e3096c6aa5f340f58b7d3b0a68711297 (127.27.77.194:44163): tablet 1649cbd8a28048b5a1019229e26bab59 (table test-table [id=c8f66b561bf14b24a22e0cb1abe1de68]) successfully deleted
W20260806 00:54:18.367501 4575 catalog_manager.cc:4808] TS e3096c6aa5f340f58b7d3b0a68711297 (127.27.77.194:44163): delete failed for tablet 1649cbd8a28048b5a1019229e26bab59 because the tablet was not found. No further retry: Not found: Tablet not found: 1649cbd8a28048b5a1019229e26bab59
W20260806 00:54:18.373134 5571 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:18.372699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:39337 (local address 127.0.0.1:60756)
0806 00:54:18.372883 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:18.372975 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:39337: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 00:54:18.373135 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 5094
W20260806 00:54:18.379822 4572 catalog_manager.cc:4584] TS 2f1e8a0fcb7d45d7a0e6e3d4cabeeeda (127.27.77.193:39337): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1649cbd8a28048b5a1019229e26bab59: Network error: Client connection negotiation failed: client connection to 127.27.77.193:39337: connect: Connection reset by peer (error 104)
I20260806 00:54:18.387692 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 5295
I20260806 00:54:18.400224 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 4412
I20260806 00:54:18.411749 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 4551
2026-08-06T00:54:18Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6328 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:54:18 dist-test-slave-sb2z krb5kdc[5639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:54:18 dist-test-slave-sb2z krb5kdc[5639](info): set up 2 sockets
Aug 06 00:54:18 dist-test-slave-sb2z krb5kdc[5639](info): commencing operation
krb5kdc: starting...
W20260806 00:54:19.655172 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:54:19 dist-test-slave-sb2z krb5kdc[5639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:54:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.254.keytab.
Entry for principal oryx/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.254.keytab.
I20260806 00:54:19.792613 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:42333
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.254.keytab
--principal=oryx/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:42333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:19.910561 5655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:19.910849 5655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:19.910892 5655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:19.914880 5655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:19.914951 5655 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 00:54:19.914969 5655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:19.914988 5655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:19.915004 5655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:19.915020 5655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:54:19.920477 5655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:42333
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.254.keytab
--principal=oryx/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:42333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:19.921581 5655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:19.922586 5655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:19.931582 5660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:19.931725 5664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:19.931725 5661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:19.931896 5655 server_base.cc:1034] running on GCE node
I20260806 00:54:19.932469 5655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:19.933630 5655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:19.934849 5655 hybrid_clock.cc:648] HybridClock initialized: now 1785977659934816 us; error 46 us; skew 500 ppm
Aug 06 00:54:19 dist-test-slave-sb2z krb5kdc[5639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977659, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:19.943620 5655 init.cc:377] Logged in from keytab as oryx/127.27.77.254@KRBTEST.COM (short username oryx)
I20260806 00:54:19.944770 5655 webserver.cc:460] Webserver started at http://127.27.77.254:36835/ using document root <none> and password file <none>
I20260806 00:54:19.945406 5655 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:19.945513 5655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:19.945767 5655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:19.948024 5655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "691e88581cf7441790575f44672dcc48"
format_stamp: "Formatted at 2026-08-06 00:54:19 on dist-test-slave-sb2z"
server_key: "24f445c58428dbe07dbf0d5d6f37055b"
server_key_iv: "c2a338fee4e3464d0afff27652e8361b"
server_key_version: "encryptionkey@0"
I20260806 00:54:19.948634 5655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "691e88581cf7441790575f44672dcc48"
format_stamp: "Formatted at 2026-08-06 00:54:19 on dist-test-slave-sb2z"
server_key: "24f445c58428dbe07dbf0d5d6f37055b"
server_key_iv: "c2a338fee4e3464d0afff27652e8361b"
server_key_version: "encryptionkey@0"
I20260806 00:54:19.952795 5655 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:54:19.955482 5672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:19.957074 5655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:54:19.957260 5655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "691e88581cf7441790575f44672dcc48"
format_stamp: "Formatted at 2026-08-06 00:54:19 on dist-test-slave-sb2z"
server_key: "24f445c58428dbe07dbf0d5d6f37055b"
server_key_iv: "c2a338fee4e3464d0afff27652e8361b"
server_key_version: "encryptionkey@0"
I20260806 00:54:19.957397 5655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:19.968216 5655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:19.971853 5655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:19.972103 5655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:19.981526 5655 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:42333
I20260806 00:54:19.981542 5764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:42333 every 8 connection(s)
I20260806 00:54:19.982585 5655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:19.985771 5765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:19.990710 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 5655
I20260806 00:54:19.990903 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:54:19.991226 27959 external_mini_cluster.cc:1428] Setting key 0ede6fefae02f1ca57952777451d2f71
I20260806 00:54:19.992540 5765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48: Bootstrap starting.
I20260806 00:54:19.995108 5765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:19.996044 5765 log.cc:824] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:19.998159 5765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48: No bootstrap required, opened a new log
Aug 06 00:54:19 dist-test-slave-sb2z krb5kdc[5639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
I20260806 00:54:20.003131 5765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "691e88581cf7441790575f44672dcc48" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42333 } }
I20260806 00:54:20.003384 5765 raft_consensus.cc:374] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:20.003450 5765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 691e88581cf7441790575f44672dcc48, State: Initialized, Role: FOLLOWER
I20260806 00:54:20.003957 5765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "691e88581cf7441790575f44672dcc48" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42333 } }
I20260806 00:54:20.004098 5765 raft_consensus.cc:388] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:20.004191 5765 raft_consensus.cc:482] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:20.004302 5765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:20.005306 5765 raft_consensus.cc:504] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "691e88581cf7441790575f44672dcc48" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42333 } }
I20260806 00:54:20.005736 5765 leader_election.cc:302] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 691e88581cf7441790575f44672dcc48; no voters:
I20260806 00:54:20.006201 5765 leader_election.cc:288] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:54:20.006285 5773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:20.006503 5773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [term 1 LEADER]: Becoming Leader. State: Replica: 691e88581cf7441790575f44672dcc48, State: Running, Role: LEADER
I20260806 00:54:20.006829 5773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "691e88581cf7441790575f44672dcc48" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42333 } }
I20260806 00:54:20.006927 5770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:19.992866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35890 (local address 127.27.77.254:42333)
0806 00:54:19.993337 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:19.993353 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:19.993392 (+ 39us) server_negotiation.cc:407] Connection header received
0806 00:54:19.994056 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:19.994073 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:19.994414 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:19.994781 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:19.995655 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:19.996934 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:19.997792 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:19.998317 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:19.998475 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.001517 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:20.001522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:20.001536 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:20.001565 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:20.003464 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.004029 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.004037 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.004047 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.004247 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.004561 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.004565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.004566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.005015 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:20.005271 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:20.005559 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 00:54:20.005919 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":144,"threads_started":1}
I20260806 00:54:20.007715 5765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:20.009723 5775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "691e88581cf7441790575f44672dcc48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "691e88581cf7441790575f44672dcc48" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42333 } } }
I20260806 00:54:20.009873 5775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:20.010262 5776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 691e88581cf7441790575f44672dcc48. Latest consensus state: current_term: 1 leader_uuid: "691e88581cf7441790575f44672dcc48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "691e88581cf7441790575f44672dcc48" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42333 } } }
I20260806 00:54:20.010601 5776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:20.010390 5791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:20.013528 5791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:20.019397 5791 catalog_manager.cc:1347] Generated new cluster ID: d4a94b31e18e4574b9403c41ace390d5
I20260806 00:54:20.019479 5791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:20.035082 5791 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:54:20.036319 5791 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:20.048180 5791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48: Generated new TSK 0
I20260806 00:54:20.048899 5791 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.193.keytab.
Entry for principal oryx/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.193.keytab.
I20260806 00:54:20.087273 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:42333
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.193.keytab
--principal=oryx/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:20.204123 5798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:20.204381 5798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:20.204427 5798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:20.208453 5798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:20.208518 5798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:20.208559 5798 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:54:20.208592 5798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:54:20.214088 5798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.193.keytab
--principal=oryx/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.77.254:42333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:20.215131 5798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:20.216135 5798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:20.226027 5806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:20.226027 5808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:20.226052 5805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:20.226500 5798 server_base.cc:1034] running on GCE node
I20260806 00:54:20.226905 5798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:20.227670 5798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:20.228855 5798 hybrid_clock.cc:648] HybridClock initialized: now 1785977660228841 us; error 30 us; skew 500 ppm
Aug 06 00:54:20 dist-test-slave-sb2z krb5kdc[5639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977660, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:20.231751 5798 init.cc:377] Logged in from keytab as oryx/127.27.77.193@KRBTEST.COM (short username oryx)
I20260806 00:54:20.232575 5798 webserver.cc:460] Webserver started at http://127.27.77.193:38619/ using document root <none> and password file <none>
I20260806 00:54:20.233096 5798 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:20.233162 5798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:20.233327 5798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:20.235321 5798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "6a44d05a654446bab11854680548637f"
format_stamp: "Formatted at 2026-08-06 00:54:20 on dist-test-slave-sb2z"
server_key: "b51b74dd5ce343a81bf33e5a824c5b62"
server_key_iv: "9cbde231c81f7947f17a9d45793c7d68"
server_key_version: "encryptionkey@0"
I20260806 00:54:20.235910 5798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "6a44d05a654446bab11854680548637f"
format_stamp: "Formatted at 2026-08-06 00:54:20 on dist-test-slave-sb2z"
server_key: "b51b74dd5ce343a81bf33e5a824c5b62"
server_key_iv: "9cbde231c81f7947f17a9d45793c7d68"
server_key_version: "encryptionkey@0"
I20260806 00:54:20.239609 5798 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:54:20.241959 5818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:20.243165 5798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:54:20.243302 5798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "6a44d05a654446bab11854680548637f"
format_stamp: "Formatted at 2026-08-06 00:54:20 on dist-test-slave-sb2z"
server_key: "b51b74dd5ce343a81bf33e5a824c5b62"
server_key_iv: "9cbde231c81f7947f17a9d45793c7d68"
server_key_version: "encryptionkey@0"
I20260806 00:54:20.243405 5798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:20.260097 5798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:20.263500 5798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:20.263785 5798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:20.265333 5798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:20.265411 5798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:20.265480 5798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:20.265527 5798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:54:20 dist-test-slave-sb2z krb5kdc[5639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977660, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
I20260806 00:54:20.277098 5798 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:43471
I20260806 00:54:20.277272 5969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:43471 every 8 connection(s)
I20260806 00:54:20.278198 5798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:54:20.280714 5770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.266221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:52667 (local address 127.27.77.254:42333)
0806 00:54:20.266352 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:20.266355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:20.267459 (+ 1104us) server_negotiation.cc:407] Connection header received
0806 00:54:20.268517 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:20.268521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:20.268581 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:20.268653 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:20.270684 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.271625 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.273078 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.273330 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.273450 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.276688 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:20.276694 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:20.276697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:20.276730 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:20.278420 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.279036 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.279042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.279046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.279153 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.279521 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.279572 (+ 51us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.279574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.279775 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:20.279888 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:20.280375 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 00:54:20.280533 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:54:20.281438 5829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.266799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:42333 (local address 127.27.77.193:52667)
0806 00:54:20.267306 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:20.267348 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:20.268249 (+ 901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:20.269138 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:20.269151 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:20.269613 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:20.270322 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.270343 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.271800 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.271803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.272928 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.272940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.273520 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.273525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.273703 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.274365 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:20.274391 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:20.276428 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:20.278540 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:20.278546 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:20.278568 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:20.278931 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:20.279261 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:20.279264 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:20.279266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:20.279409 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:20.279886 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:20.279893 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:20.280135 (+ 242us) client_negotiation.cc:769] Sending connection context
0806 00:54:20.280318 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 00:54:20.280643 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":142,"threads_started":1}
I20260806 00:54:20.282703 5971 heartbeater.cc:347] Connected to a master server at 127.27.77.254:42333
I20260806 00:54:20.282864 5971 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:20.283486 5971 heartbeater.cc:511] Master 127.27.77.254:42333 requested a full tablet report, sending...
I20260806 00:54:20.284776 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 5798
I20260806 00:54:20.284942 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:54:20.285377 27959 external_mini_cluster.cc:1428] Setting key 9f315ef776c9698231d91470a8667148
I20260806 00:54:20.286131 5698 ts_manager.cc:194] Registered new tserver with Master: 6a44d05a654446bab11854680548637f (127.27.77.193:43471)
I20260806 00:54:20.287837 5698 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:52667
I20260806 00:54:20.294194 5980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.285409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:42333 (local address 127.27.77.193:37411)
0806 00:54:20.285734 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:20.285750 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:20.285866 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:20.286219 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:20.286223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:20.286441 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:20.286564 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.286571 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.287869 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.287875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.288929 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.288944 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.289617 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.289619 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.289690 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.290224 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:20.290246 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:20.290576 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:20.292469 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:20.292475 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:20.292480 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:20.292775 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:20.293062 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:20.293068 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:20.293072 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:20.293183 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:20.293707 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:20.293709 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:20.293799 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 00:54:20.293909 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 00:54:20.294034 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":2,"thread_start_us":156,"threads_started":1}
I20260806 00:54:20.294399 5770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.285863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:37411 (local address 127.27.77.254:42333)
0806 00:54:20.285984 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:20.285988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:20.286005 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:54:20.286042 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:20.286045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:20.286088 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:20.286181 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:20.286697 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.287718 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.289083 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.289417 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.289480 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.290706 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:20.290714 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:20.290721 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:20.290755 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:20.292345 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.292878 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.292881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.292882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.292954 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.293280 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.293282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.293283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.293559 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:20.293708 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:20.294008 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 00:54:20.294154 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":53}
WARNING: no policy specified for oryx/127.27.77.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.77.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.194.keytab.
Entry for principal oryx/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.194.keytab.
I20260806 00:54:20.319063 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:42333
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.194.keytab
--principal=oryx/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:20.438730 5983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:20.438983 5983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:20.439028 5983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:20.442961 5983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:20.443020 5983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:20.443060 5983 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:54:20.443094 5983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:54:20.448509 5983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.194.keytab
--principal=oryx/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.77.254:42333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:20.449523 5983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:20.450525 5983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:20.460419 5990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:20.460410 5989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:20.460422 5992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:20.461094 5983 server_base.cc:1034] running on GCE node
I20260806 00:54:20.461474 5983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:20.462073 5983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:20.463279 5983 hybrid_clock.cc:648] HybridClock initialized: now 1785977660463259 us; error 41 us; skew 500 ppm
Aug 06 00:54:20 dist-test-slave-sb2z krb5kdc[5639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977660, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:20.466267 5983 init.cc:377] Logged in from keytab as oryx/127.27.77.194@KRBTEST.COM (short username oryx)
I20260806 00:54:20.467216 5983 webserver.cc:460] Webserver started at http://127.27.77.194:38125/ using document root <none> and password file <none>
I20260806 00:54:20.467873 5983 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:20.467968 5983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:20.468209 5983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:20.470275 5983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "28a43b0c75d14bd3b4d96545e01d59c1"
format_stamp: "Formatted at 2026-08-06 00:54:20 on dist-test-slave-sb2z"
server_key: "10741fe6bb1adb422029a286da5b8190"
server_key_iv: "2091c0faa8e8252890cc9907de742fa3"
server_key_version: "encryptionkey@0"
I20260806 00:54:20.470852 5983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "28a43b0c75d14bd3b4d96545e01d59c1"
format_stamp: "Formatted at 2026-08-06 00:54:20 on dist-test-slave-sb2z"
server_key: "10741fe6bb1adb422029a286da5b8190"
server_key_iv: "2091c0faa8e8252890cc9907de742fa3"
server_key_version: "encryptionkey@0"
I20260806 00:54:20.474745 5983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 00:54:20.477304 6002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:20.478632 5983 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:54:20.478884 5983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "28a43b0c75d14bd3b4d96545e01d59c1"
format_stamp: "Formatted at 2026-08-06 00:54:20 on dist-test-slave-sb2z"
server_key: "10741fe6bb1adb422029a286da5b8190"
server_key_iv: "2091c0faa8e8252890cc9907de742fa3"
server_key_version: "encryptionkey@0"
I20260806 00:54:20.479007 5983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:20.496573 5983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:20.499814 5983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:20.500027 5983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:20.501596 5983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:20.501673 5983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:20.501747 5983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:20.501801 5983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:54:20 dist-test-slave-sb2z krb5kdc[5639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977660, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
I20260806 00:54:20.513753 5983 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:39969
I20260806 00:54:20.513957 6142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:39969 every 8 connection(s)
I20260806 00:54:20.514932 5983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:54:20.516647 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 5983
I20260806 00:54:20.516758 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:54:20.517163 27959 external_mini_cluster.cc:1428] Setting key 3a5e35cc9130f1680a0388acf071abba
I20260806 00:54:20.517745 5770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.502533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:47135 (local address 127.27.77.254:42333)
0806 00:54:20.502705 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:20.502711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:20.503699 (+ 988us) server_negotiation.cc:407] Connection header received
0806 00:54:20.505004 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:20.505009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:20.505080 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:20.505194 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:20.506763 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.507461 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.509212 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.509654 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.509784 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.513154 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:20.513158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:20.513161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:20.513196 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:20.515131 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.515909 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.515915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.515919 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.516025 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.516462 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.516465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.516466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.516684 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:20.516812 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:20.517434 (+ 622us) server_negotiation.cc:299] Negotiation successful
0806 00:54:20.517569 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 00:54:20.518707 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.502974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:42333 (local address 127.27.77.194:47135)
0806 00:54:20.503488 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:20.503583 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:20.504665 (+ 1082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:20.505411 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:20.505418 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:20.505899 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:20.506598 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.506616 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.507658 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.507662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.509021 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.509033 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.509868 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.509876 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.510079 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.510847 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:20.510865 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:20.512878 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:20.515279 (+ 2401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:20.515289 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:20.515309 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:20.515757 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:20.516152 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:20.516155 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:20.516156 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:20.516307 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:20.516874 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:20.516881 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:20.517139 (+ 258us) client_negotiation.cc:769] Sending connection context
0806 00:54:20.517357 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 00:54:20.517722 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":136,"threads_started":1}
I20260806 00:54:20.521348 6143 heartbeater.cc:347] Connected to a master server at 127.27.77.254:42333
I20260806 00:54:20.521503 6143 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:20.522083 6143 heartbeater.cc:511] Master 127.27.77.254:42333 requested a full tablet report, sending...
I20260806 00:54:20.523190 5697 ts_manager.cc:194] Registered new tserver with Master: 28a43b0c75d14bd3b4d96545e01d59c1 (127.27.77.194:39969)
I20260806 00:54:20.524034 5697 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:47135
WARNING: no policy specified for oryx/127.27.77.195@KRBTEST.COM; defaulting to no policy
I20260806 00:54:20.531301 5770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.523824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:37935 (local address 127.27.77.254:42333)
0806 00:54:20.523962 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:20.523967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:20.524205 (+ 238us) server_negotiation.cc:407] Connection header received
0806 00:54:20.524364 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:20.524370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:20.524419 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:20.524489 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:20.525054 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.525567 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.526413 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.526643 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.526753 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.527951 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:20.527955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:20.527958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:20.527983 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:20.529562 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.530016 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.530024 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.530025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.530100 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.530393 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.530397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.530399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.530626 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:20.530742 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:20.531015 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 00:54:20.531138 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:54:20.531312 6154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.523660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:42333 (local address 127.27.77.194:37935)
0806 00:54:20.524125 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:20.524140 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:20.524276 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:20.524601 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:20.524604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:20.524821 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:20.524928 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.524934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.525711 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.525715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.526285 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.526293 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.526768 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.526772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.526873 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.527401 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:20.527413 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:20.527810 (+ 397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:20.529692 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:20.529696 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:20.529698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:20.529902 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:20.530197 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:20.530200 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:20.530202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:20.530294 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:20.530733 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:20.530736 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:20.530842 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 00:54:20.530956 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 00:54:20.531130 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":100,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.77.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.195.keytab.
Entry for principal oryx/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.195.keytab.
I20260806 00:54:20.551024 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:42333
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.195.keytab
--principal=oryx/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:20.673061 6156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:20.673329 6156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:20.673411 6156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:20.677433 6156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:20.677495 6156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:20.677572 6156 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:54:20.677660 6156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:54:20.683161 6156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.195.keytab
--principal=oryx/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.77.254:42333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:20.684393 6156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:20.685478 6156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:20.695436 6164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:20.695468 6168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:20.695515 6165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:20.695608 6156 server_base.cc:1034] running on GCE node
I20260806 00:54:20.696233 6156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:20.696902 6156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:20.698091 6156 hybrid_clock.cc:648] HybridClock initialized: now 1785977660698065 us; error 57 us; skew 500 ppm
Aug 06 00:54:20 dist-test-slave-sb2z krb5kdc[5639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977660, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:20.701236 6156 init.cc:377] Logged in from keytab as oryx/127.27.77.195@KRBTEST.COM (short username oryx)
I20260806 00:54:20.702224 6156 webserver.cc:460] Webserver started at http://127.27.77.195:46119/ using document root <none> and password file <none>
I20260806 00:54:20.702823 6156 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:20.702919 6156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:20.703173 6156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:20.705403 6156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "dc0e00981ffc442da6ea52aef3e5a701"
format_stamp: "Formatted at 2026-08-06 00:54:20 on dist-test-slave-sb2z"
server_key: "fd1514883164ecd54bbfff19c71697ec"
server_key_iv: "8d359249b5878ee7593af283c5781dd5"
server_key_version: "encryptionkey@0"
I20260806 00:54:20.706014 6156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "dc0e00981ffc442da6ea52aef3e5a701"
format_stamp: "Formatted at 2026-08-06 00:54:20 on dist-test-slave-sb2z"
server_key: "fd1514883164ecd54bbfff19c71697ec"
server_key_iv: "8d359249b5878ee7593af283c5781dd5"
server_key_version: "encryptionkey@0"
I20260806 00:54:20.710433 6156 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:54:20.713090 6177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:20.714366 6156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:54:20.714514 6156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "dc0e00981ffc442da6ea52aef3e5a701"
format_stamp: "Formatted at 2026-08-06 00:54:20 on dist-test-slave-sb2z"
server_key: "fd1514883164ecd54bbfff19c71697ec"
server_key_iv: "8d359249b5878ee7593af283c5781dd5"
server_key_version: "encryptionkey@0"
I20260806 00:54:20.714637 6156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:20.742352 6156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:20.745590 6156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:20.745859 6156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:20.747699 6156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:20.747774 6156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:20.747844 6156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:20.747875 6156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:54:20 dist-test-slave-sb2z krb5kdc[5639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977660, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
I20260806 00:54:20.759608 6156 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:42217
I20260806 00:54:20.759862 6320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:42217 every 8 connection(s)
I20260806 00:54:20.760800 6156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:54:20.763833 5770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.748606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:41161 (local address 127.27.77.254:42333)
0806 00:54:20.748744 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:20.748748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:20.749871 (+ 1123us) server_negotiation.cc:407] Connection header received
0806 00:54:20.751023 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:20.751027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:20.751096 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:20.751162 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:20.752833 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.753482 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.754673 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.754927 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.755065 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.758591 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:20.758608 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:20.758611 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:20.758641 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:20.761165 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.762189 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.762193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.762195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.762297 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.762666 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.762669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.762671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.762872 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:20.762993 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:20.763494 (+ 501us) server_negotiation.cc:299] Negotiation successful
0806 00:54:20.763659 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:54:20.764590 6187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.749166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:42333 (local address 127.27.77.195:41161)
0806 00:54:20.749701 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:20.749746 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:20.750759 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:20.751682 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:20.751692 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:20.752072 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:20.752674 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.752690 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.753617 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.753621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.754501 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.754512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.755151 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.755158 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.755416 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.756145 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:20.756167 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:20.758377 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:20.761306 (+ 2929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:20.761312 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:20.761327 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:20.761684 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:20.762382 (+ 698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:20.762385 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:20.762387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:20.762562 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:20.762993 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:20.763001 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:20.763255 (+ 254us) client_negotiation.cc:769] Sending connection context
0806 00:54:20.763450 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 00:54:20.763791 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":152,"threads_started":1}
I20260806 00:54:20.765774 6321 heartbeater.cc:347] Connected to a master server at 127.27.77.254:42333
I20260806 00:54:20.765944 6321 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:20.766561 6321 heartbeater.cc:511] Master 127.27.77.254:42333 requested a full tablet report, sending...
I20260806 00:54:20.767772 5698 ts_manager.cc:194] Registered new tserver with Master: dc0e00981ffc442da6ea52aef3e5a701 (127.27.77.195:42217)
I20260806 00:54:20.768570 5698 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:41161
I20260806 00:54:20.768738 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 6156
I20260806 00:54:20.768894 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:54:20.769284 27959 external_mini_cluster.cc:1428] Setting key d73f3ea21b4ec6ff6195d533ed3cbdc6
I20260806 00:54:20.771382 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:54:20.776188 6327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.768211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:42333 (local address 127.27.77.195:37225)
0806 00:54:20.768645 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:20.768663 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:20.768814 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:20.769185 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:20.769187 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:20.769442 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:20.769587 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.769597 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.770585 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.770588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.771145 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.771153 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.771724 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.771732 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.771850 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.772383 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:20.772397 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:20.772715 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:20.774619 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:20.774623 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:20.774625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:20.774830 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:20.775125 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:20.775129 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:20.775131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:20.775220 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:20.775733 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:20.775736 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:20.775820 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 00:54:20.775913 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 00:54:20.776028 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":103,"threads_started":1}
I20260806 00:54:20.776381 5770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.768299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:37225 (local address 127.27.77.254:42333)
0806 00:54:20.768418 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:20.768422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:20.768739 (+ 317us) server_negotiation.cc:407] Connection header received
0806 00:54:20.768933 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:20.768936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:20.768985 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:20.769039 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:20.769718 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.770443 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.771268 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.771593 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.771693 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.772863 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:20.772870 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:20.772876 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:20.772910 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:20.774493 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.774931 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.774935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.774937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.775023 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.775313 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.775319 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.775324 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.775617 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:20.775743 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:20.776002 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 00:54:20.776149 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 00:54:20 dist-test-slave-sb2z krb5kdc[5639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977660, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 00:54:20 dist-test-slave-sb2z krb5kdc[5639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:20 dist-test-slave-sb2z krb5kdc[5639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 00:54:20.798385 5770 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:20.791944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35896 (local address 127.27.77.254:42333)
0806 00:54:20.792113 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:20.792117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:20.792132 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:54:20.792174 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:20.792176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:20.792217 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:20.792340 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:20.793125 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.793662 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.794544 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.794763 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.794873 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.798252 (+ 3379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35896: BlockingRecv error: recv got EOF from 127.0.0.1:35896 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 00:54:20 dist-test-slave-sb2z krb5kdc[5639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977660, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
I20260806 00:54:20.807516 5770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.799847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35900 (local address 127.27.77.254:42333)
0806 00:54:20.799986 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:20.799991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:20.800123 (+ 132us) server_negotiation.cc:407] Connection header received
0806 00:54:20.800191 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:20.800193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:20.800234 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:20.800298 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:20.800869 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.801349 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.802084 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.802281 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.802352 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.804685 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:20.804688 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:20.804690 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:20.804715 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:20.805959 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.806332 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.806335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.806336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.806408 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:20.806731 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:20.806733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:20.806734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:20.806899 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:20.807003 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:20.807213 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 00:54:20.807304 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:54:20.810817 5698 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:54:20.813925 5698 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:54:20.830145 6362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.823058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:43471 (local address 127.0.0.1:32980)
0806 00:54:20.824703 (+ 1645us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:20.824758 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:20.824910 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:20.825474 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:20.825481 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:20.825505 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:20.825628 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.825634 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.827099 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.827103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.828627 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.828640 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.829710 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.829713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.829801 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.829827 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:54:20.829869 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 00:54:20.829985 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1458,"mutex_wait_us":87,"spinlock_wait_cycles":25600,"thread_start_us":98,"threads_started":1}
I20260806 00:54:20.830178 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.824322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58188 (local address 127.27.77.194:39969)
0806 00:54:20.825423 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:20.825436 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:20.825460 (+ 24us) server_negotiation.cc:407] Connection header received
0806 00:54:20.825543 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:20.825547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:20.825720 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:20.825876 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:20.826063 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.827194 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.828780 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.829441 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.829561 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.829750 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:20.829955 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 00:54:20.830050 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":991,"thread_start_us":109,"threads_started":1}
I20260806 00:54:20.830221 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.822988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36104 (local address 127.27.77.195:42217)
0806 00:54:20.824212 (+ 1224us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:20.824235 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:20.824861 (+ 626us) server_negotiation.cc:407] Connection header received
0806 00:54:20.825128 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:20.825138 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:20.825348 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:20.825486 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:20.825958 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.827202 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.828391 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.829106 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.829220 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.829388 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:20.829940 (+ 552us) server_negotiation.cc:299] Negotiation successful
0806 00:54:20.830056 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1099,"thread_start_us":113,"threads_started":1}
I20260806 00:54:20.830632 6357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.822878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:42217 (local address 127.0.0.1:36104)
0806 00:54:20.824703 (+ 1825us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:20.824758 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:20.824910 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:20.825681 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:20.825684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:20.825704 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:20.825825 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.825833 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.827353 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.827361 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.828255 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.828265 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.829240 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.829243 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.829353 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.829400 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 00:54:20.830361 (+ 961us) client_negotiation.cc:240] Negotiation successful
0806 00:54:20.830436 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1602,"mutex_wait_us":3,"spinlock_wait_cycles":2560,"thread_start_us":113,"threads_started":1}
I20260806 00:54:20.830868 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.823223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32980 (local address 127.27.77.193:43471)
0806 00:54:20.823724 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:20.823738 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:20.824911 (+ 1173us) server_negotiation.cc:407] Connection header received
0806 00:54:20.825057 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:20.825061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:20.825297 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:20.825754 (+ 457us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:20.825909 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.826979 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.828765 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.829571 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.830382 (+ 811us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.830545 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:20.830601 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 00:54:20.830701 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":98,"threads_started":1}
I20260806 00:54:20.831744 6365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.824081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:39969 (local address 127.0.0.1:58188)
0806 00:54:20.825303 (+ 1222us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:20.825317 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:20.825419 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:20.825865 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:20.825868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:20.825891 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:20.825967 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.825973 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.827316 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.827319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.828627 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.828640 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.829581 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.829585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.829694 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.829720 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:54:20.829822 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 00:54:20.831619 (+ 1797us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1154,"mutex_wait_us":2650,"thread_start_us":86,"threads_started":1}
I20260806 00:54:20.833603 6242 tablet_service.cc:1467] Processing CreateTablet for tablet df6bb9fa37584f58a94ca934f8706a0d (DEFAULT_TABLE table=test-table [id=e01ddc743ed34e64a52d894a76b1a255]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:20.833616 5881 tablet_service.cc:1467] Processing CreateTablet for tablet df6bb9fa37584f58a94ca934f8706a0d (DEFAULT_TABLE table=test-table [id=e01ddc743ed34e64a52d894a76b1a255]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:20.834712 6242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df6bb9fa37584f58a94ca934f8706a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:20.834774 5881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df6bb9fa37584f58a94ca934f8706a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:20.834472 6059 tablet_service.cc:1467] Processing CreateTablet for tablet df6bb9fa37584f58a94ca934f8706a0d (DEFAULT_TABLE table=test-table [id=e01ddc743ed34e64a52d894a76b1a255]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:20.835418 6059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df6bb9fa37584f58a94ca934f8706a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:20.841609 6366 tablet_bootstrap.cc:492] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701: Bootstrap starting.
I20260806 00:54:20.843060 6367 tablet_bootstrap.cc:492] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f: Bootstrap starting.
I20260806 00:54:20.843111 6369 tablet_bootstrap.cc:492] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1: Bootstrap starting.
I20260806 00:54:20.844748 6366 tablet_bootstrap.cc:654] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:20.845211 6367 tablet_bootstrap.cc:654] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:20.845259 6369 tablet_bootstrap.cc:654] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:20.845721 6366 log.cc:824] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:20.845969 6367 log.cc:824] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:20.846081 6369 log.cc:824] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:20.848021 6366 tablet_bootstrap.cc:492] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701: No bootstrap required, opened a new log
I20260806 00:54:20.848027 6367 tablet_bootstrap.cc:492] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f: No bootstrap required, opened a new log
I20260806 00:54:20.848067 6369 tablet_bootstrap.cc:492] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1: No bootstrap required, opened a new log
I20260806 00:54:20.848240 6367 ts_tablet_manager.cc:1397] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:54:20.848237 6369 ts_tablet_manager.cc:1397] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:54:20.848238 6366 ts_tablet_manager.cc:1397] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:54:20.852226 6367 raft_consensus.cc:348] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } }
I20260806 00:54:20.852497 6367 raft_consensus.cc:374] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:20.852316 6366 raft_consensus.cc:348] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } }
I20260806 00:54:20.852316 6369 raft_consensus.cc:348] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } }
I20260806 00:54:20.852555 6367 raft_consensus.cc:729] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a44d05a654446bab11854680548637f, State: Initialized, Role: FOLLOWER
I20260806 00:54:20.852567 6369 raft_consensus.cc:374] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:20.852567 6366 raft_consensus.cc:374] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:20.852648 6366 raft_consensus.cc:729] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0e00981ffc442da6ea52aef3e5a701, State: Initialized, Role: FOLLOWER
I20260806 00:54:20.852643 6369 raft_consensus.cc:729] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28a43b0c75d14bd3b4d96545e01d59c1, State: Initialized, Role: FOLLOWER
I20260806 00:54:20.853039 6367 consensus_queue.cc:260] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } }
I20260806 00:54:20.853089 6369 consensus_queue.cc:260] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } }
I20260806 00:54:20.853089 6366 consensus_queue.cc:260] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } }
I20260806 00:54:20.853757 5971 heartbeater.cc:503] Master 127.27.77.254:42333 was elected leader, sending a full tablet report...
I20260806 00:54:20.854094 6367 ts_tablet_manager.cc:1428] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:54:20.854115 6366 ts_tablet_manager.cc:1428] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260806 00:54:20.854480 6321 heartbeater.cc:503] Master 127.27.77.254:42333 was elected leader, sending a full tablet report...
I20260806 00:54:20.854741 6143 heartbeater.cc:503] Master 127.27.77.254:42333 was elected leader, sending a full tablet report...
I20260806 00:54:20.855687 6369 ts_tablet_manager.cc:1428] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260806 00:54:20.892678 6375 raft_consensus.cc:482] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:54:20.892935 6375 raft_consensus.cc:504] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } }
I20260806 00:54:20.894148 6375 leader_election.cc:288] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc0e00981ffc442da6ea52aef3e5a701 (127.27.77.195:42217), 6a44d05a654446bab11854680548637f (127.27.77.193:43471)
I20260806 00:54:20.898280 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.894459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:43471 (local address 127.27.77.194:36275)
0806 00:54:20.894608 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:20.894629 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:20.894735 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:20.895017 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:20.895020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:20.895047 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:20.895225 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.895235 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.896112 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.896115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.897121 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.897133 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.897857 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.897859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.897948 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.897968 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:54:20.898028 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 00:54:20.898085 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":1}
I20260806 00:54:20.898278 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.894591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:36275 (local address 127.27.77.193:43471)
0806 00:54:20.894752 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:20.894757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:20.894776 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:54:20.894841 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:20.894844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:20.894900 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:20.894986 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:20.895366 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.895990 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.897259 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.897740 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.897862 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.897917 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:20.898067 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 00:54:20.898144 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 00:54:20.898932 5909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df6bb9fa37584f58a94ca934f8706a0d" candidate_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a44d05a654446bab11854680548637f" is_pre_election: true
I20260806 00:54:20.899070 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.894752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:36607 (local address 127.27.77.195:42217)
0806 00:54:20.894891 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:20.894895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:20.895132 (+ 237us) server_negotiation.cc:407] Connection header received
0806 00:54:20.895204 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:20.895206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:20.895246 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:20.895343 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:20.896051 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.896576 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.897718 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.898281 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.898402 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.898454 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:20.898807 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 00:54:20.898890 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:54:20.899282 5909 raft_consensus.cc:2454] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28a43b0c75d14bd3b4d96545e01d59c1 in term 0.
I20260806 00:54:20.899463 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.894473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:42217 (local address 127.27.77.194:36607)
0806 00:54:20.895021 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:20.895034 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:20.895126 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:20.895379 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:20.895381 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:20.895397 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:20.895921 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.895930 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.896693 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.896697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.897585 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:20.897601 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.898542 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.898546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:20.898649 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.898680 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 00:54:20.899201 (+ 521us) client_negotiation.cc:240] Negotiation successful
0806 00:54:20.899299 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":45,"thread_start_us":160,"threads_started":1}
I20260806 00:54:20.899844 6006 leader_election.cc:302] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28a43b0c75d14bd3b4d96545e01d59c1, 6a44d05a654446bab11854680548637f; no voters:
I20260806 00:54:20.900151 6375 raft_consensus.cc:2781] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:54:20.900239 6375 raft_consensus.cc:482] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:54:20.900295 6375 raft_consensus.cc:3037] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:20.900385 6266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df6bb9fa37584f58a94ca934f8706a0d" candidate_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0e00981ffc442da6ea52aef3e5a701" is_pre_election: true
I20260806 00:54:20.900660 6266 raft_consensus.cc:2454] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28a43b0c75d14bd3b4d96545e01d59c1 in term 0.
I20260806 00:54:20.901602 6375 raft_consensus.cc:504] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } }
I20260806 00:54:20.901991 6375 leader_election.cc:288] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [CANDIDATE]: Term 1 election: Requested vote from peers dc0e00981ffc442da6ea52aef3e5a701 (127.27.77.195:42217), 6a44d05a654446bab11854680548637f (127.27.77.193:43471)
I20260806 00:54:20.902346 5909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df6bb9fa37584f58a94ca934f8706a0d" candidate_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a44d05a654446bab11854680548637f"
I20260806 00:54:20.902344 6266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df6bb9fa37584f58a94ca934f8706a0d" candidate_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0e00981ffc442da6ea52aef3e5a701"
I20260806 00:54:20.902482 6266 raft_consensus.cc:3037] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:20.902482 5909 raft_consensus.cc:3037] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:20.904080 5909 raft_consensus.cc:2454] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28a43b0c75d14bd3b4d96545e01d59c1 in term 1.
I20260806 00:54:20.904240 6266 raft_consensus.cc:2454] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28a43b0c75d14bd3b4d96545e01d59c1 in term 1.
I20260806 00:54:20.904498 6006 leader_election.cc:302] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28a43b0c75d14bd3b4d96545e01d59c1, 6a44d05a654446bab11854680548637f; no voters:
I20260806 00:54:20.904681 6375 raft_consensus.cc:2781] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:20.904901 6375 raft_consensus.cc:686] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 LEADER]: Becoming Leader. State: Replica: 28a43b0c75d14bd3b4d96545e01d59c1, State: Running, Role: LEADER
I20260806 00:54:20.905278 6375 consensus_queue.cc:237] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } }
I20260806 00:54:20.908814 5696 catalog_manager.cc:5495] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28a43b0c75d14bd3b4d96545e01d59c1 (127.27.77.194). New cstate: current_term: 1 leader_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } health_report { overall_health: HEALTHY } } }
I20260806 00:54:20.927641 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:20.924180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58192 (local address 127.27.77.194:39969)
0806 00:54:20.924334 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:20.924338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:20.924352 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:54:20.924432 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:20.924434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:20.924488 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:20.924588 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:54:20.924807 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.925363 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.926203 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:20.926423 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:20.926519 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:20.926789 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:54:20.927226 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:54:20.927323 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:20.927403 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 00:54:20.927459 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 00:54:20.934080 5909 raft_consensus.cc:1261] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Refusing update from remote peer 28a43b0c75d14bd3b4d96545e01d59c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:20.934087 6266 raft_consensus.cc:1261] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Refusing update from remote peer 28a43b0c75d14bd3b4d96545e01d59c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:20.934962 6377 consensus_queue.cc:1035] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:20.935216 6375 consensus_queue.cc:1035] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:20.943827 6380 mvcc.cc:205] Tried to move back new op lower bound from 7315364499176919040 to 7315364499070992384. Current Snapshot: MvccSnapshot[applied={T|T < 7315364499176919040}]
I20260806 00:54:20.944758 6381 mvcc.cc:205] Tried to move back new op lower bound from 7315364499176919040 to 7315364499070992384. Current Snapshot: MvccSnapshot[applied={T|T < 7315364499176919040}]
I20260806 00:54:20.945802 6382 mvcc.cc:205] Tried to move back new op lower bound from 7315364499176919040 to 7315364499070992384. Current Snapshot: MvccSnapshot[applied={T|T < 7315364499176919040}]
I20260806 00:54:20.953855 5696 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:20.954010 5696 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:20.956746 5696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48: Sending DeleteTablet for 3 replicas of tablet df6bb9fa37584f58a94ca934f8706a0d
I20260806 00:54:20.957581 5881 tablet_service.cc:1514] Processing DeleteTablet for tablet df6bb9fa37584f58a94ca934f8706a0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:20 UTC) from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:32980
I20260806 00:54:20.957582 6059 tablet_service.cc:1514] Processing DeleteTablet for tablet df6bb9fa37584f58a94ca934f8706a0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:20 UTC) from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:58188
I20260806 00:54:20.958151 6393 tablet_replica.cc:330] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1: stopping tablet replica
I20260806 00:54:20.958137 6242 tablet_service.cc:1514] Processing DeleteTablet for tablet df6bb9fa37584f58a94ca934f8706a0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:20 UTC) from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:36104
I20260806 00:54:20.958464 6392 tablet_replica.cc:330] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f: stopping tablet replica
I20260806 00:54:20.958801 6392 raft_consensus.cc:2229] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:20.958830 6394 tablet_replica.cc:330] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701: stopping tablet replica
I20260806 00:54:20.959139 6392 raft_consensus.cc:2258] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:20.959146 6394 raft_consensus.cc:2229] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:20.959125 5696 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:54:20.959580 5696 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:54:20.959607 6393 raft_consensus.cc:2229] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:54:20.960009 6393 raft_consensus.cc:2258] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:20.960728 6394 raft_consensus.cc:2258] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:20.961634 6392 ts_tablet_manager.cc:1905] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:20.962800 6394 ts_tablet_manager.cc:1905] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:20.964613 6392 ts_tablet_manager.cc:1918] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:20.964722 6392 log.cc:1196] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/wal/wals/df6bb9fa37584f58a94ca934f8706a0d
I20260806 00:54:20.964911 5881 tablet_service.cc:1467] Processing CreateTablet for tablet 04f0b1d59b11452e8b0852a4d7b2ddcb (DEFAULT_TABLE table=test-table [id=5189f557f4d94d7295dca5f2ec5e3f1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:20.965102 6392 ts_tablet_manager.cc:1939] T df6bb9fa37584f58a94ca934f8706a0d P 6a44d05a654446bab11854680548637f: Deleting consensus metadata
I20260806 00:54:20.965204 5881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04f0b1d59b11452e8b0852a4d7b2ddcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:20.965335 6059 tablet_service.cc:1467] Processing CreateTablet for tablet 04f0b1d59b11452e8b0852a4d7b2ddcb (DEFAULT_TABLE table=test-table [id=5189f557f4d94d7295dca5f2ec5e3f1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:20.965373 6242 tablet_service.cc:1467] Processing CreateTablet for tablet 04f0b1d59b11452e8b0852a4d7b2ddcb (DEFAULT_TABLE table=test-table [id=5189f557f4d94d7295dca5f2ec5e3f1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:20.965626 6059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04f0b1d59b11452e8b0852a4d7b2ddcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:20.965649 6242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04f0b1d59b11452e8b0852a4d7b2ddcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:20.966355 5678 catalog_manager.cc:4841] TS 6a44d05a654446bab11854680548637f (127.27.77.193:43471): tablet df6bb9fa37584f58a94ca934f8706a0d (table test-table [id=e01ddc743ed34e64a52d894a76b1a255]) successfully deleted
I20260806 00:54:20.967868 6393 ts_tablet_manager.cc:1905] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:20.968171 6367 tablet_bootstrap.cc:492] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f: Bootstrap starting.
I20260806 00:54:20.968209 6366 tablet_bootstrap.cc:492] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701: Bootstrap starting.
I20260806 00:54:20.969200 6366 tablet_bootstrap.cc:654] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:20.969200 6367 tablet_bootstrap.cc:654] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:20.969846 6369 tablet_bootstrap.cc:492] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1: Bootstrap starting.
I20260806 00:54:20.970319 6393 ts_tablet_manager.cc:1918] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:20.970423 6393 log.cc:1196] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/df6bb9fa37584f58a94ca934f8706a0d
I20260806 00:54:20.970746 6393 ts_tablet_manager.cc:1939] T df6bb9fa37584f58a94ca934f8706a0d P 28a43b0c75d14bd3b4d96545e01d59c1: Deleting consensus metadata
I20260806 00:54:20.970899 6369 tablet_bootstrap.cc:654] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:20.970961 6367 tablet_bootstrap.cc:492] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f: No bootstrap required, opened a new log
I20260806 00:54:20.971040 6367 ts_tablet_manager.cc:1397] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:54:20.971899 5676 catalog_manager.cc:4841] TS 28a43b0c75d14bd3b4d96545e01d59c1 (127.27.77.194:39969): tablet df6bb9fa37584f58a94ca934f8706a0d (table test-table [id=e01ddc743ed34e64a52d894a76b1a255]) successfully deleted
I20260806 00:54:20.971908 6367 raft_consensus.cc:348] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:20.972066 6367 raft_consensus.cc:374] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:20.972106 6367 raft_consensus.cc:729] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a44d05a654446bab11854680548637f, State: Initialized, Role: FOLLOWER
I20260806 00:54:20.972237 6367 consensus_queue.cc:260] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:20.972528 6367 ts_tablet_manager.cc:1428] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:54:20.973464 6366 tablet_bootstrap.cc:492] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701: No bootstrap required, opened a new log
I20260806 00:54:20.973553 6366 ts_tablet_manager.cc:1397] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260806 00:54:20.974069 6366 raft_consensus.cc:348] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:20.974187 6366 raft_consensus.cc:374] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:20.974223 6366 raft_consensus.cc:729] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0e00981ffc442da6ea52aef3e5a701, State: Initialized, Role: FOLLOWER
I20260806 00:54:20.974354 6366 consensus_queue.cc:260] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:20.974462 6394 ts_tablet_manager.cc:1918] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:20.974543 6394 log.cc:1196] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/df6bb9fa37584f58a94ca934f8706a0d
I20260806 00:54:20.974669 6366 ts_tablet_manager.cc:1428] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:54:20.974880 6394 ts_tablet_manager.cc:1939] T df6bb9fa37584f58a94ca934f8706a0d P dc0e00981ffc442da6ea52aef3e5a701: Deleting consensus metadata
I20260806 00:54:20.976051 6369 tablet_bootstrap.cc:492] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1: No bootstrap required, opened a new log
I20260806 00:54:20.976145 6369 ts_tablet_manager.cc:1397] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260806 00:54:20.976573 6369 raft_consensus.cc:348] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:20.976692 6369 raft_consensus.cc:374] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:20.976727 6369 raft_consensus.cc:729] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28a43b0c75d14bd3b4d96545e01d59c1, State: Initialized, Role: FOLLOWER
I20260806 00:54:20.976850 6369 consensus_queue.cc:260] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:20.977010 5676 catalog_manager.cc:4841] TS dc0e00981ffc442da6ea52aef3e5a701 (127.27.77.195:42217): tablet df6bb9fa37584f58a94ca934f8706a0d (table test-table [id=e01ddc743ed34e64a52d894a76b1a255]) successfully deleted
I20260806 00:54:20.977129 6369 ts_tablet_manager.cc:1428] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260806 00:54:21.011951 6322 tablet.cc:2369] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:54:21.016266 6145 tablet.cc:2369] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:54:21.029709 5972 tablet.cc:2369] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:54:21.261230 6386 raft_consensus.cc:482] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:54:21.261422 6386 raft_consensus.cc:504] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.261926 6386 leader_election.cc:288] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a44d05a654446bab11854680548637f (127.27.77.193:43471), dc0e00981ffc442da6ea52aef3e5a701 (127.27.77.195:42217)
I20260806 00:54:21.262506 6266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04f0b1d59b11452e8b0852a4d7b2ddcb" candidate_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0e00981ffc442da6ea52aef3e5a701" is_pre_election: true
I20260806 00:54:21.262497 5909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04f0b1d59b11452e8b0852a4d7b2ddcb" candidate_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a44d05a654446bab11854680548637f" is_pre_election: true
I20260806 00:54:21.262699 5909 raft_consensus.cc:2454] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28a43b0c75d14bd3b4d96545e01d59c1 in term 0.
I20260806 00:54:21.262699 6266 raft_consensus.cc:2454] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28a43b0c75d14bd3b4d96545e01d59c1 in term 0.
I20260806 00:54:21.263039 6006 leader_election.cc:302] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28a43b0c75d14bd3b4d96545e01d59c1, 6a44d05a654446bab11854680548637f; no voters:
I20260806 00:54:21.263276 6386 raft_consensus.cc:2781] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:54:21.263350 6386 raft_consensus.cc:482] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:54:21.263410 6386 raft_consensus.cc:3037] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:21.264815 6386 raft_consensus.cc:504] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.265223 6386 leader_election.cc:288] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [CANDIDATE]: Term 1 election: Requested vote from peers 6a44d05a654446bab11854680548637f (127.27.77.193:43471), dc0e00981ffc442da6ea52aef3e5a701 (127.27.77.195:42217)
I20260806 00:54:21.265615 5909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04f0b1d59b11452e8b0852a4d7b2ddcb" candidate_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a44d05a654446bab11854680548637f"
I20260806 00:54:21.265722 5909 raft_consensus.cc:3037] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:21.266734 5909 raft_consensus.cc:2454] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28a43b0c75d14bd3b4d96545e01d59c1 in term 1.
I20260806 00:54:21.266997 6266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04f0b1d59b11452e8b0852a4d7b2ddcb" candidate_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0e00981ffc442da6ea52aef3e5a701"
I20260806 00:54:21.267078 6006 leader_election.cc:302] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28a43b0c75d14bd3b4d96545e01d59c1, 6a44d05a654446bab11854680548637f; no voters:
I20260806 00:54:21.267192 6266 raft_consensus.cc:3037] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:21.267256 6386 raft_consensus.cc:2781] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:21.267340 6386 raft_consensus.cc:686] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 LEADER]: Becoming Leader. State: Replica: 28a43b0c75d14bd3b4d96545e01d59c1, State: Running, Role: LEADER
I20260806 00:54:21.267491 6386 consensus_queue.cc:237] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.268453 6266 raft_consensus.cc:2454] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28a43b0c75d14bd3b4d96545e01d59c1 in term 1.
I20260806 00:54:21.269186 5698 catalog_manager.cc:5495] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28a43b0c75d14bd3b4d96545e01d59c1 (127.27.77.194). New cstate: current_term: 1 leader_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } health_report { overall_health: UNKNOWN } } }
Aug 06 00:54:21 dist-test-slave-sb2z krb5kdc[5639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977659, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.254@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
I20260806 00:54:21.366781 6408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.357449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:42333 (local address 127.0.0.1:45896)
0806 00:54:21.357912 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:21.357929 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:21.358139 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:21.358412 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:21.358415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:21.358874 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:21.358983 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:21.358990 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.359850 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.359856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:21.360421 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:21.360429 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.360885 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.360888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:21.360991 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.361718 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:21.361731 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:21.363364 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:21.365038 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:21.365047 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:21.365055 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:21.365347 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:21.365652 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:21.365655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:21.365657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:21.365742 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:21.366157 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:21.366164 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:21.366280 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 00:54:21.366398 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 00:54:21.366573 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":143,"threads_started":1}
I20260806 00:54:21.366781 6409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.357598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45896 (local address 127.27.77.254:42333)
0806 00:54:21.357884 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:21.357895 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:21.358068 (+ 173us) server_negotiation.cc:407] Connection header received
0806 00:54:21.358244 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:21.358247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:21.358293 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:21.358394 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:21.359118 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.359742 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.360546 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.360771 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.360883 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.363511 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:21.363515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:21.363518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:21.363577 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:21.364914 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:21.365455 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:21.365461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:21.365463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:21.365549 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:21.365839 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:21.365844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:21.365846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:21.366030 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:21.366137 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:21.366479 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 00:54:21.366634 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":205,"thread_start_us":98,"threads_started":1}
I20260806 00:54:21.371184 5698 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:45896:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 00:54:21.371970 5698 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:54:21.377158 5881 tablet_service.cc:1467] Processing CreateTablet for tablet 34cbe2b826a4496487dfdd5ac65e6391 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=515808b134fc48b9ba42c5f3f879e115]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:54:21.377252 6242 tablet_service.cc:1467] Processing CreateTablet for tablet 34cbe2b826a4496487dfdd5ac65e6391 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=515808b134fc48b9ba42c5f3f879e115]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:54:21.377621 5881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34cbe2b826a4496487dfdd5ac65e6391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:21.377619 6242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34cbe2b826a4496487dfdd5ac65e6391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:21.379963 6059 tablet_service.cc:1467] Processing CreateTablet for tablet 34cbe2b826a4496487dfdd5ac65e6391 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=515808b134fc48b9ba42c5f3f879e115]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:54:21.380424 6059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34cbe2b826a4496487dfdd5ac65e6391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:21.380788 6366 tablet_bootstrap.cc:492] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701: Bootstrap starting.
I20260806 00:54:21.381631 6367 tablet_bootstrap.cc:492] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f: Bootstrap starting.
I20260806 00:54:21.381795 6366 tablet_bootstrap.cc:654] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:21.382575 6367 tablet_bootstrap.cc:654] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:21.383212 6366 tablet_bootstrap.cc:492] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701: No bootstrap required, opened a new log
I20260806 00:54:21.383303 6366 ts_tablet_manager.cc:1397] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 00:54:21.383870 6366 raft_consensus.cc:348] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.384008 6366 raft_consensus.cc:374] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:21.384066 6366 raft_consensus.cc:729] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0e00981ffc442da6ea52aef3e5a701, State: Initialized, Role: FOLLOWER
I20260806 00:54:21.384094 6367 tablet_bootstrap.cc:492] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f: No bootstrap required, opened a new log
I20260806 00:54:21.384186 6367 ts_tablet_manager.cc:1397] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:54:21.384197 6366 consensus_queue.cc:260] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.384595 6366 ts_tablet_manager.cc:1428] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 00:54:21.384523 6374 tablet_replica.cc:439] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } }
I20260806 00:54:21.384683 6374 tablet_replica.cc:442] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:54:21.384694 6367 raft_consensus.cc:348] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.384836 6367 raft_consensus.cc:374] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:21.384896 6367 raft_consensus.cc:729] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a44d05a654446bab11854680548637f, State: Initialized, Role: FOLLOWER
I20260806 00:54:21.385030 6367 consensus_queue.cc:260] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.385311 6369 tablet_bootstrap.cc:492] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1: Bootstrap starting.
I20260806 00:54:21.386286 6369 tablet_bootstrap.cc:654] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:21.387001 6367 ts_tablet_manager.cc:1428] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:54:21.386945 6373 tablet_replica.cc:439] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } }
I20260806 00:54:21.387128 6373 tablet_replica.cc:442] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:54:21.387667 6369 tablet_bootstrap.cc:492] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1: No bootstrap required, opened a new log
I20260806 00:54:21.387764 6369 ts_tablet_manager.cc:1397] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 00:54:21.388235 6369 raft_consensus.cc:348] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.388357 6369 raft_consensus.cc:374] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:21.388408 6369 raft_consensus.cc:729] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28a43b0c75d14bd3b4d96545e01d59c1, State: Initialized, Role: FOLLOWER
I20260806 00:54:21.388553 6369 consensus_queue.cc:260] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.388849 6386 tablet_replica.cc:439] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } }
I20260806 00:54:21.389003 6386 tablet_replica.cc:442] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:54:21.388861 6369 ts_tablet_manager.cc:1428] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:54:21.428821 6373 raft_consensus.cc:482] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:54:21.428960 6373 raft_consensus.cc:504] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.429998 6373 leader_election.cc:288] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28a43b0c75d14bd3b4d96545e01d59c1 (127.27.77.194:39969), dc0e00981ffc442da6ea52aef3e5a701 (127.27.77.195:42217)
I20260806 00:54:21.433866 6386 raft_consensus.cc:482] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:54:21.433987 6386 raft_consensus.cc:504] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.434115 6417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.430297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:39969 (local address 127.27.77.193:41715)
0806 00:54:21.430597 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:21.430612 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:21.430748 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:21.431102 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:21.431104 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:21.431124 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:21.431241 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:21.431247 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.432067 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.432071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:21.432948 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:21.432957 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.433633 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.433637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:21.433724 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.433753 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 00:54:21.433846 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 00:54:21.433926 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":21,"thread_start_us":95,"threads_started":1}
I20260806 00:54:21.434479 6386 leader_election.cc:288] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a44d05a654446bab11854680548637f (127.27.77.193:43471), dc0e00981ffc442da6ea52aef3e5a701 (127.27.77.195:42217)
I20260806 00:54:21.434849 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.430381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:41715 (local address 127.27.77.194:39969)
0806 00:54:21.430688 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:21.430693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:21.430710 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:54:21.430906 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:21.430908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:21.430952 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:21.431088 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:21.431364 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.431948 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.433081 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.433509 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.433643 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.433701 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:21.434638 (+ 937us) server_negotiation.cc:299] Negotiation successful
0806 00:54:21.434723 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":83,"threads_started":1}
I20260806 00:54:21.434844 5909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34cbe2b826a4496487dfdd5ac65e6391" candidate_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a44d05a654446bab11854680548637f" is_pre_election: true
I20260806 00:54:21.434854 6266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34cbe2b826a4496487dfdd5ac65e6391" candidate_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0e00981ffc442da6ea52aef3e5a701" is_pre_election: true
I20260806 00:54:21.434989 5909 raft_consensus.cc:2454] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28a43b0c75d14bd3b4d96545e01d59c1 in term 0.
I20260806 00:54:21.435009 6266 raft_consensus.cc:2454] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28a43b0c75d14bd3b4d96545e01d59c1 in term 0.
I20260806 00:54:21.435257 6066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34cbe2b826a4496487dfdd5ac65e6391" candidate_uuid: "6a44d05a654446bab11854680548637f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" is_pre_election: true
I20260806 00:54:21.435302 6006 leader_election.cc:302] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28a43b0c75d14bd3b4d96545e01d59c1, 6a44d05a654446bab11854680548637f; no voters:
I20260806 00:54:21.435511 6066 raft_consensus.cc:2454] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a44d05a654446bab11854680548637f in term 0.
I20260806 00:54:21.435592 6419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.430739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:42217 (local address 127.27.77.193:55483)
0806 00:54:21.431131 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:21.431146 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:21.431245 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:21.431550 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:21.431554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:21.431580 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:21.431699 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:21.431704 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.432475 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.432478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:21.433341 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:21.433350 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.434319 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.434324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:21.434432 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.434458 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:54:21.434539 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 00:54:21.435342 (+ 803us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":39,"thread_start_us":112,"threads_started":1}
I20260806 00:54:21.435667 6386 raft_consensus.cc:2781] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:54:21.435734 6386 raft_consensus.cc:482] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:54:21.435760 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.430909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:55483 (local address 127.27.77.195:42217)
0806 00:54:21.431199 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:21.431203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:21.431218 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:54:21.431377 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:21.431380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:21.431423 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:21.431513 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:21.431816 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.432340 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.433471 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.434008 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.434141 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.434203 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:21.434660 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 00:54:21.434749 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":83,"threads_started":1}
I20260806 00:54:21.435783 6386 raft_consensus.cc:3037] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:21.436030 5819 leader_election.cc:302] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28a43b0c75d14bd3b4d96545e01d59c1, 6a44d05a654446bab11854680548637f; no voters:
I20260806 00:54:21.436309 6373 raft_consensus.cc:2781] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:54:21.436376 6373 raft_consensus.cc:482] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:54:21.436431 6373 raft_consensus.cc:3037] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:21.436681 6266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34cbe2b826a4496487dfdd5ac65e6391" candidate_uuid: "6a44d05a654446bab11854680548637f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0e00981ffc442da6ea52aef3e5a701" is_pre_election: true
I20260806 00:54:21.436821 6266 raft_consensus.cc:2454] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a44d05a654446bab11854680548637f in term 0.
I20260806 00:54:21.437215 6386 raft_consensus.cc:504] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.437614 6386 leader_election.cc:288] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [CANDIDATE]: Term 1 election: Requested vote from peers 6a44d05a654446bab11854680548637f (127.27.77.193:43471), dc0e00981ffc442da6ea52aef3e5a701 (127.27.77.195:42217)
I20260806 00:54:21.437649 6373 raft_consensus.cc:504] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.437980 6266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34cbe2b826a4496487dfdd5ac65e6391" candidate_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0e00981ffc442da6ea52aef3e5a701"
I20260806 00:54:21.437992 5909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34cbe2b826a4496487dfdd5ac65e6391" candidate_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a44d05a654446bab11854680548637f"
I20260806 00:54:21.438064 6373 leader_election.cc:288] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [CANDIDATE]: Term 1 election: Requested vote from peers 28a43b0c75d14bd3b4d96545e01d59c1 (127.27.77.194:39969), dc0e00981ffc442da6ea52aef3e5a701 (127.27.77.195:42217)
I20260806 00:54:21.438090 6266 raft_consensus.cc:3037] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:21.438186 5909 raft_consensus.cc:2379] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 28a43b0c75d14bd3b4d96545e01d59c1 in current term 1: Already voted for candidate 6a44d05a654446bab11854680548637f in this term.
I20260806 00:54:21.438421 6066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34cbe2b826a4496487dfdd5ac65e6391" candidate_uuid: "6a44d05a654446bab11854680548637f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a43b0c75d14bd3b4d96545e01d59c1"
I20260806 00:54:21.438463 6265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34cbe2b826a4496487dfdd5ac65e6391" candidate_uuid: "6a44d05a654446bab11854680548637f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0e00981ffc442da6ea52aef3e5a701"
I20260806 00:54:21.438552 6066 raft_consensus.cc:2379] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a44d05a654446bab11854680548637f in current term 1: Already voted for candidate 28a43b0c75d14bd3b4d96545e01d59c1 in this term.
I20260806 00:54:21.438993 5819 leader_election.cc:302] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a44d05a654446bab11854680548637f; no voters: 28a43b0c75d14bd3b4d96545e01d59c1, dc0e00981ffc442da6ea52aef3e5a701
I20260806 00:54:21.439179 6373 raft_consensus.cc:2726] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 00:54:21.439307 6266 raft_consensus.cc:2454] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28a43b0c75d14bd3b4d96545e01d59c1 in term 1.
I20260806 00:54:21.439729 6003 leader_election.cc:302] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28a43b0c75d14bd3b4d96545e01d59c1, dc0e00981ffc442da6ea52aef3e5a701; no voters: 6a44d05a654446bab11854680548637f
I20260806 00:54:21.439891 6386 raft_consensus.cc:2781] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:21.439982 6386 raft_consensus.cc:686] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 LEADER]: Becoming Leader. State: Replica: 28a43b0c75d14bd3b4d96545e01d59c1, State: Running, Role: LEADER
I20260806 00:54:21.440151 6386 consensus_queue.cc:237] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.440877 6375 tablet_replica.cc:439] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1: TxnStatusTablet state changed. Reason: New leader 28a43b0c75d14bd3b4d96545e01d59c1. Latest consensus state: current_term: 1 leader_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } }
I20260806 00:54:21.441002 6375 tablet_replica.cc:442] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:54:21.441259 6422 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 00:54:21.441464 6422 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 00:54:21.442487 5696 catalog_manager.cc:5495] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28a43b0c75d14bd3b4d96545e01d59c1 (127.27.77.194). New cstate: current_term: 1 leader_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } health_report { overall_health: UNKNOWN } } }
I20260806 00:54:21.466850 6408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.463308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:39969 (local address 127.0.0.1:57014)
0806 00:54:21.463509 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:21.463526 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:21.463718 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:21.464051 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:21.464054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:21.464061 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:54:21.464179 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:21.464185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.464937 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.464941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:21.465547 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:21.465555 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.466050 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.466054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:21.466153 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.466198 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:54:21.466528 (+ 330us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:54:21.466547 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 00:54:21.466626 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 00:54:21.466720 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 00:54:21.466850 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.463422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57014 (local address 127.27.77.194:39969)
0806 00:54:21.463575 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:21.463582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:21.463603 (+ 21us) server_negotiation.cc:407] Connection header received
0806 00:54:21.463886 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:21.463889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:21.463939 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:21.464025 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:54:21.464293 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.464814 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.465687 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.465912 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.466034 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.466360 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:54:21.466436 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:54:21.466520 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:21.466639 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 00:54:21.466694 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:54:21.469674 5909 raft_consensus.cc:1261] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Refusing update from remote peer 28a43b0c75d14bd3b4d96545e01d59c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:21.469774 6266 raft_consensus.cc:1261] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Refusing update from remote peer 28a43b0c75d14bd3b4d96545e01d59c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:21.470268 6386 consensus_queue.cc:1035] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:21.470444 6375 consensus_queue.cc:1035] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:21.471855 6373 tablet_replica.cc:439] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f: TxnStatusTablet state changed. Reason: New leader 28a43b0c75d14bd3b4d96545e01d59c1. Latest consensus state: current_term: 1 leader_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } }
I20260806 00:54:21.471973 6373 tablet_replica.cc:442] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:54:21.472257 6374 tablet_replica.cc:439] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701: TxnStatusTablet state changed. Reason: New leader 28a43b0c75d14bd3b4d96545e01d59c1. Latest consensus state: current_term: 1 leader_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } }
I20260806 00:54:21.472373 6374 tablet_replica.cc:442] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:54:21.473943 6375 tablet_replica.cc:439] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } }
I20260806 00:54:21.474054 6375 tablet_replica.cc:442] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:54:21.474967 6374 tablet_replica.cc:439] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } }
I20260806 00:54:21.475088 6374 tablet_replica.cc:442] T 34cbe2b826a4496487dfdd5ac65e6391 P dc0e00981ffc442da6ea52aef3e5a701: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:54:21.475490 6375 tablet_replica.cc:439] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } }
I20260806 00:54:21.476042 6375 tablet_replica.cc:442] T 34cbe2b826a4496487dfdd5ac65e6391 P 28a43b0c75d14bd3b4d96545e01d59c1: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:54:21.476135 6373 tablet_replica.cc:439] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } } }
I20260806 00:54:21.476248 6373 tablet_replica.cc:442] T 34cbe2b826a4496487dfdd5ac65e6391 P 6a44d05a654446bab11854680548637f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:54:21.491338 6440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.487729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:39969 (local address 127.27.77.194:48537)
0806 00:54:21.487991 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:21.488004 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:21.488113 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:21.488349 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:21.488351 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:21.488357 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:54:21.488445 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:21.488449 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.489419 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.489421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:21.490047 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:21.490055 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.490639 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.490642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:21.490702 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.490744 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:54:21.491025 (+ 281us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:54:21.491030 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 00:54:21.491079 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 00:54:21.491192 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":19,"thread_start_us":71,"threads_started":1}
I20260806 00:54:21.491345 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.487809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:48537 (local address 127.27.77.194:39969)
0806 00:54:21.487937 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:21.487943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:21.488071 (+ 128us) server_negotiation.cc:407] Connection header received
0806 00:54:21.488201 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:21.488204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:21.488253 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:21.488333 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:54:21.488542 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.489319 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.490168 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.490519 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.490634 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.490866 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:54:21.490944 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:54:21.491014 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:21.491093 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 00:54:21.491183 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":47}
I20260806 00:54:21.498605 5909 raft_consensus.cc:1261] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Refusing update from remote peer 28a43b0c75d14bd3b4d96545e01d59c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:21.498692 6266 raft_consensus.cc:1261] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Refusing update from remote peer 28a43b0c75d14bd3b4d96545e01d59c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:21.499080 6375 consensus_queue.cc:1035] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:21.499285 6386 consensus_queue.cc:1035] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:21.541330 5696 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:21.541501 5696 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:21.543696 5696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48: Sending DeleteTablet for 3 replicas of tablet 04f0b1d59b11452e8b0852a4d7b2ddcb
I20260806 00:54:21.544253 5881 tablet_service.cc:1514] Processing DeleteTablet for tablet 04f0b1d59b11452e8b0852a4d7b2ddcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:21 UTC) from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:32980
I20260806 00:54:21.544414 6242 tablet_service.cc:1514] Processing DeleteTablet for tablet 04f0b1d59b11452e8b0852a4d7b2ddcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:21 UTC) from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:36104
I20260806 00:54:21.544742 6445 tablet_replica.cc:330] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701: stopping tablet replica
I20260806 00:54:21.544910 6445 raft_consensus.cc:2229] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:21.545012 6058 tablet_service.cc:1514] Processing DeleteTablet for tablet 04f0b1d59b11452e8b0852a4d7b2ddcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:21 UTC) from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:58188
I20260806 00:54:21.545145 6445 raft_consensus.cc:2258] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:21.545861 6445 ts_tablet_manager.cc:1905] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:21.549209 6445 ts_tablet_manager.cc:1918] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:54:21.549381 6445 log.cc:1196] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/04f0b1d59b11452e8b0852a4d7b2ddcb
I20260806 00:54:21.549746 6445 ts_tablet_manager.cc:1939] T 04f0b1d59b11452e8b0852a4d7b2ddcb P dc0e00981ffc442da6ea52aef3e5a701: Deleting consensus metadata
I20260806 00:54:21.550588 5676 catalog_manager.cc:4841] TS dc0e00981ffc442da6ea52aef3e5a701 (127.27.77.195:42217): tablet 04f0b1d59b11452e8b0852a4d7b2ddcb (table test-table [id=5189f557f4d94d7295dca5f2ec5e3f1f]) successfully deleted
I20260806 00:54:21.551756 6444 tablet_replica.cc:330] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f: stopping tablet replica
I20260806 00:54:21.551751 6448 tablet_replica.cc:330] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1: stopping tablet replica
I20260806 00:54:21.551898 6448 raft_consensus.cc:2229] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:54:21.551899 6444 raft_consensus.cc:2229] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:21.552027 6444 raft_consensus.cc:2258] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:21.552114 6448 raft_consensus.cc:2258] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:21.552773 6448 ts_tablet_manager.cc:1905] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:21.555754 6448 ts_tablet_manager.cc:1918] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:54:21.555845 6448 log.cc:1196] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/04f0b1d59b11452e8b0852a4d7b2ddcb
I20260806 00:54:21.556121 6448 ts_tablet_manager.cc:1939] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 28a43b0c75d14bd3b4d96545e01d59c1: Deleting consensus metadata
I20260806 00:54:21.556185 6444 ts_tablet_manager.cc:1905] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:21.556869 5676 catalog_manager.cc:4841] TS 28a43b0c75d14bd3b4d96545e01d59c1 (127.27.77.194:39969): tablet 04f0b1d59b11452e8b0852a4d7b2ddcb (table test-table [id=5189f557f4d94d7295dca5f2ec5e3f1f]) successfully deleted
I20260806 00:54:21.559288 6444 ts_tablet_manager.cc:1918] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:54:21.559401 6444 log.cc:1196] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/wal/wals/04f0b1d59b11452e8b0852a4d7b2ddcb
I20260806 00:54:21.559741 6444 ts_tablet_manager.cc:1939] T 04f0b1d59b11452e8b0852a4d7b2ddcb P 6a44d05a654446bab11854680548637f: Deleting consensus metadata
I20260806 00:54:21.560467 5678 catalog_manager.cc:4841] TS 6a44d05a654446bab11854680548637f (127.27.77.193:43471): tablet 04f0b1d59b11452e8b0852a4d7b2ddcb (table test-table [id=5189f557f4d94d7295dca5f2ec5e3f1f]) successfully deleted
Aug 06 00:54:21 dist-test-slave-sb2z krb5kdc[5639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 00:54:21 dist-test-slave-sb2z krb5kdc[5639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:21 dist-test-slave-sb2z krb5kdc[5639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 00:54:21.580734 6409 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:21.573940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45902 (local address 127.27.77.254:42333)
0806 00:54:21.574087 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:21.574093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:21.574302 (+ 209us) server_negotiation.cc:407] Connection header received
0806 00:54:21.574492 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:21.574494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:21.574539 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:21.574635 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:21.575362 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.576010 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.576906 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.577155 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.577268 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.580594 (+ 3326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45902: BlockingRecv error: recv got EOF from 127.0.0.1:45902 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 00:54:21 dist-test-slave-sb2z krb5kdc[5639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM
I20260806 00:54:21.591254 6409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.582340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45906 (local address 127.27.77.254:42333)
0806 00:54:21.582454 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:21.582460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:21.582559 (+ 99us) server_negotiation.cc:407] Connection header received
0806 00:54:21.582779 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:21.582782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:21.582832 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:21.582920 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:21.583716 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.584263 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.585054 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.585285 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.585394 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.587955 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:21.587960 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:21.587963 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:21.587992 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:21.589601 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:21.590012 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:21.590016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:21.590018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:21.590114 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:21.590382 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:21.590386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:21.590388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:21.590621 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:21.590757 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:21.590963 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 00:54:21.591091 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:54:21.593989 5696 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:54:21.594445 5696 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:54:21.599244 5881 tablet_service.cc:1467] Processing CreateTablet for tablet d1ea9593d95b49bbb0b4ad4a2a008b31 (DEFAULT_TABLE table=test-table [id=d281dacfaf884da19af57dbe558a8702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:21.599582 5881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1ea9593d95b49bbb0b4ad4a2a008b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:21.599447 6242 tablet_service.cc:1467] Processing CreateTablet for tablet d1ea9593d95b49bbb0b4ad4a2a008b31 (DEFAULT_TABLE table=test-table [id=d281dacfaf884da19af57dbe558a8702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:21.599835 6242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1ea9593d95b49bbb0b4ad4a2a008b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:21.601780 6366 tablet_bootstrap.cc:492] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701: Bootstrap starting.
I20260806 00:54:21.602063 6367 tablet_bootstrap.cc:492] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f: Bootstrap starting.
I20260806 00:54:21.602025 6058 tablet_service.cc:1467] Processing CreateTablet for tablet d1ea9593d95b49bbb0b4ad4a2a008b31 (DEFAULT_TABLE table=test-table [id=d281dacfaf884da19af57dbe558a8702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:21.602387 6058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1ea9593d95b49bbb0b4ad4a2a008b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:21.602766 6366 tablet_bootstrap.cc:654] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:21.603231 6367 tablet_bootstrap.cc:654] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:21.604548 6367 tablet_bootstrap.cc:492] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f: No bootstrap required, opened a new log
I20260806 00:54:21.604647 6367 ts_tablet_manager.cc:1397] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 00:54:21.604882 6369 tablet_bootstrap.cc:492] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1: Bootstrap starting.
I20260806 00:54:21.605088 6367 raft_consensus.cc:348] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.605195 6367 raft_consensus.cc:374] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:21.605223 6367 raft_consensus.cc:729] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a44d05a654446bab11854680548637f, State: Initialized, Role: FOLLOWER
I20260806 00:54:21.605335 6367 consensus_queue.cc:260] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.605741 6369 tablet_bootstrap.cc:654] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:21.606987 6366 tablet_bootstrap.cc:492] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701: No bootstrap required, opened a new log
I20260806 00:54:21.607051 6366 ts_tablet_manager.cc:1397] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 00:54:21.607659 6366 raft_consensus.cc:348] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.607700 6367 ts_tablet_manager.cc:1428] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260806 00:54:21.607750 6366 raft_consensus.cc:374] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:21.607772 6366 raft_consensus.cc:729] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0e00981ffc442da6ea52aef3e5a701, State: Initialized, Role: FOLLOWER
I20260806 00:54:21.607913 6366 consensus_queue.cc:260] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.608193 6366 ts_tablet_manager.cc:1428] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:54:21.609004 6369 tablet_bootstrap.cc:492] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1: No bootstrap required, opened a new log
I20260806 00:54:21.609104 6369 ts_tablet_manager.cc:1397] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 00:54:21.609527 6369 raft_consensus.cc:348] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.609653 6369 raft_consensus.cc:374] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:21.609695 6369 raft_consensus.cc:729] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28a43b0c75d14bd3b4d96545e01d59c1, State: Initialized, Role: FOLLOWER
I20260806 00:54:21.609820 6369 consensus_queue.cc:260] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.610126 6369 ts_tablet_manager.cc:1428] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 00:54:21.655484 6374 raft_consensus.cc:482] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:54:21.655759 6374 raft_consensus.cc:504] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.656915 6374 leader_election.cc:288] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a44d05a654446bab11854680548637f (127.27.77.193:43471), 28a43b0c75d14bd3b4d96545e01d59c1 (127.27.77.194:39969)
I20260806 00:54:21.661151 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.657240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:53507 (local address 127.27.77.193:43471)
0806 00:54:21.657567 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:21.657579 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:21.657670 (+ 91us) server_negotiation.cc:407] Connection header received
0806 00:54:21.657847 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:21.657850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:21.657907 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:21.658006 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:21.658301 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.658873 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.660058 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.660537 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.660686 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.660746 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:21.660931 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 00:54:21.661012 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":131,"threads_started":1}
I20260806 00:54:21.661150 6468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.657164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:43471 (local address 127.27.77.195:53507)
0806 00:54:21.657558 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:21.657575 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:21.657727 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:21.658019 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:21.658022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:21.658039 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:21.658163 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:21.658176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.659001 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.659005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:21.659927 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:21.659937 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.660676 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.660680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:21.660776 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.660800 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 00:54:21.660900 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 00:54:21.660984 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":136,"threads_started":1}
I20260806 00:54:21.661593 5909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ea9593d95b49bbb0b4ad4a2a008b31" candidate_uuid: "dc0e00981ffc442da6ea52aef3e5a701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a44d05a654446bab11854680548637f" is_pre_election: true
I20260806 00:54:21.661778 5909 raft_consensus.cc:2454] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0e00981ffc442da6ea52aef3e5a701 in term 0.
I20260806 00:54:21.661917 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.657174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:39969 (local address 127.27.77.195:50171)
0806 00:54:21.657716 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:21.657729 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:21.657869 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:21.658205 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:21.658208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:21.658227 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:21.658847 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:21.658859 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.659760 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.659763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:21.660591 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:21.660601 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.661452 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.661458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:21.661564 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.661586 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:54:21.661681 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 00:54:21.661752 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":111,"threads_started":1}
I20260806 00:54:21.661999 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.657336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:50171 (local address 127.27.77.194:39969)
0806 00:54:21.657503 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:21.657508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:21.657829 (+ 321us) server_negotiation.cc:407] Connection header received
0806 00:54:21.658001 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:21.658004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:21.658092 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:21.658190 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:21.658987 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.659603 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.660731 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.661280 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.661421 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.661471 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:21.661738 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 00:54:21.661840 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 00:54:21.662218 6180 leader_election.cc:302] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a44d05a654446bab11854680548637f, dc0e00981ffc442da6ea52aef3e5a701; no voters:
I20260806 00:54:21.662497 6374 raft_consensus.cc:2781] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:54:21.662472 6066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ea9593d95b49bbb0b4ad4a2a008b31" candidate_uuid: "dc0e00981ffc442da6ea52aef3e5a701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" is_pre_election: true
I20260806 00:54:21.662559 6374 raft_consensus.cc:482] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:54:21.662607 6374 raft_consensus.cc:3037] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:21.662618 6066 raft_consensus.cc:2454] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0e00981ffc442da6ea52aef3e5a701 in term 0.
I20260806 00:54:21.663849 6374 raft_consensus.cc:504] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.664233 6374 leader_election.cc:288] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [CANDIDATE]: Term 1 election: Requested vote from peers 6a44d05a654446bab11854680548637f (127.27.77.193:43471), 28a43b0c75d14bd3b4d96545e01d59c1 (127.27.77.194:39969)
I20260806 00:54:21.664577 6066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ea9593d95b49bbb0b4ad4a2a008b31" candidate_uuid: "dc0e00981ffc442da6ea52aef3e5a701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a43b0c75d14bd3b4d96545e01d59c1"
I20260806 00:54:21.664644 5909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1ea9593d95b49bbb0b4ad4a2a008b31" candidate_uuid: "dc0e00981ffc442da6ea52aef3e5a701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a44d05a654446bab11854680548637f"
I20260806 00:54:21.664714 6066 raft_consensus.cc:3037] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:21.664805 5909 raft_consensus.cc:3037] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:21.665827 5909 raft_consensus.cc:2454] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0e00981ffc442da6ea52aef3e5a701 in term 1.
I20260806 00:54:21.665870 6066 raft_consensus.cc:2454] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0e00981ffc442da6ea52aef3e5a701 in term 1.
I20260806 00:54:21.666141 6180 leader_election.cc:302] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a44d05a654446bab11854680548637f, dc0e00981ffc442da6ea52aef3e5a701; no voters:
I20260806 00:54:21.666400 6374 raft_consensus.cc:2781] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:21.666679 6374 raft_consensus.cc:686] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [term 1 LEADER]: Becoming Leader. State: Replica: dc0e00981ffc442da6ea52aef3e5a701, State: Running, Role: LEADER
I20260806 00:54:21.666953 6374 consensus_queue.cc:237] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.669498 5696 catalog_manager.cc:5495] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc0e00981ffc442da6ea52aef3e5a701 (127.27.77.195). New cstate: current_term: 1 leader_uuid: "dc0e00981ffc442da6ea52aef3e5a701" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } health_report { overall_health: HEALTHY } } }
I20260806 00:54:21.691905 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:21.687884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39704 (local address 127.27.77.195:42217)
0806 00:54:21.688041 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:21.688046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:21.688063 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:54:21.688290 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:21.688293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:21.688354 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:21.688477 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:54:21.688752 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.689314 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.690287 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:21.690526 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:21.690658 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:21.690981 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:54:21.691458 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:54:21.691643 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:21.691689 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 00:54:21.691750 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 00:54:21.694607 5909 raft_consensus.cc:1261] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Refusing update from remote peer dc0e00981ffc442da6ea52aef3e5a701: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:21.694828 6066 raft_consensus.cc:1261] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Refusing update from remote peer dc0e00981ffc442da6ea52aef3e5a701: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:21.695163 6374 consensus_queue.cc:1035] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:21.695416 6471 consensus_queue.cc:1035] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:21.707957 5696 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:21.708132 5696 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:21.710403 5696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48: Sending DeleteTablet for 3 replicas of tablet d1ea9593d95b49bbb0b4ad4a2a008b31
I20260806 00:54:21.711054 5881 tablet_service.cc:1514] Processing DeleteTablet for tablet d1ea9593d95b49bbb0b4ad4a2a008b31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:21 UTC) from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:32980
I20260806 00:54:21.711082 6242 tablet_service.cc:1514] Processing DeleteTablet for tablet d1ea9593d95b49bbb0b4ad4a2a008b31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:21 UTC) from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:36104
I20260806 00:54:21.711275 6444 tablet_replica.cc:330] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f: stopping tablet replica
I20260806 00:54:21.711297 6058 tablet_service.cc:1514] Processing DeleteTablet for tablet d1ea9593d95b49bbb0b4ad4a2a008b31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:21 UTC) from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:58188
I20260806 00:54:21.711409 6444 raft_consensus.cc:2229] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:21.711491 6445 tablet_replica.cc:330] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701: stopping tablet replica
I20260806 00:54:21.711642 6445 raft_consensus.cc:2229] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:54:21.711920 6444 raft_consensus.cc:2258] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:21.711959 6445 raft_consensus.cc:2258] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:21.712157 6448 tablet_replica.cc:330] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1: stopping tablet replica
I20260806 00:54:21.712087 5696 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 00:54:21.712311 6448 raft_consensus.cc:2229] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:21.712438 6448 raft_consensus.cc:2258] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 00:54:21.712563 5696 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:54:21.713165 6444 ts_tablet_manager.cc:1905] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:21.713311 6448 ts_tablet_manager.cc:1905] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:21.715382 6444 ts_tablet_manager.cc:1918] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:21.715444 6444 log.cc:1196] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-0/wal/wals/d1ea9593d95b49bbb0b4ad4a2a008b31
I20260806 00:54:21.715775 6444 ts_tablet_manager.cc:1939] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 6a44d05a654446bab11854680548637f: Deleting consensus metadata
I20260806 00:54:21.715960 6448 ts_tablet_manager.cc:1918] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:21.716030 6448 log.cc:1196] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/d1ea9593d95b49bbb0b4ad4a2a008b31
I20260806 00:54:21.716287 6448 ts_tablet_manager.cc:1939] T d1ea9593d95b49bbb0b4ad4a2a008b31 P 28a43b0c75d14bd3b4d96545e01d59c1: Deleting consensus metadata
I20260806 00:54:21.716455 5678 catalog_manager.cc:4841] TS 6a44d05a654446bab11854680548637f (127.27.77.193:43471): tablet d1ea9593d95b49bbb0b4ad4a2a008b31 (table test-table [id=d281dacfaf884da19af57dbe558a8702]) successfully deleted
I20260806 00:54:21.716952 5676 catalog_manager.cc:4841] TS 28a43b0c75d14bd3b4d96545e01d59c1 (127.27.77.194:39969): tablet d1ea9593d95b49bbb0b4ad4a2a008b31 (table test-table [id=d281dacfaf884da19af57dbe558a8702]) successfully deleted
I20260806 00:54:21.717207 6445 ts_tablet_manager.cc:1905] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:21.720165 6445 ts_tablet_manager.cc:1918] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:21.720255 6445 log.cc:1196] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/d1ea9593d95b49bbb0b4ad4a2a008b31
I20260806 00:54:21.721338 5881 tablet_service.cc:1467] Processing CreateTablet for tablet f64eae87430e4db8b72830afe36748af (DEFAULT_TABLE table=test-table [id=5db0309f27154f5789656be477489b37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:21.721517 6445 ts_tablet_manager.cc:1939] T d1ea9593d95b49bbb0b4ad4a2a008b31 P dc0e00981ffc442da6ea52aef3e5a701: Deleting consensus metadata
I20260806 00:54:21.721639 5881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64eae87430e4db8b72830afe36748af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:21.722272 5676 catalog_manager.cc:4841] TS dc0e00981ffc442da6ea52aef3e5a701 (127.27.77.195:42217): tablet d1ea9593d95b49bbb0b4ad4a2a008b31 (table test-table [id=d281dacfaf884da19af57dbe558a8702]) successfully deleted
I20260806 00:54:21.722774 6242 tablet_service.cc:1467] Processing CreateTablet for tablet f64eae87430e4db8b72830afe36748af (DEFAULT_TABLE table=test-table [id=5db0309f27154f5789656be477489b37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:21.723073 6242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64eae87430e4db8b72830afe36748af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:21.723990 6367 tablet_bootstrap.cc:492] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f: Bootstrap starting.
I20260806 00:54:21.725183 6367 tablet_bootstrap.cc:654] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:21.725476 6366 tablet_bootstrap.cc:492] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701: Bootstrap starting.
I20260806 00:54:21.725711 6058 tablet_service.cc:1467] Processing CreateTablet for tablet f64eae87430e4db8b72830afe36748af (DEFAULT_TABLE table=test-table [id=5db0309f27154f5789656be477489b37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:21.726003 6058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64eae87430e4db8b72830afe36748af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:21.726514 6366 tablet_bootstrap.cc:654] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:21.727856 6366 tablet_bootstrap.cc:492] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701: No bootstrap required, opened a new log
I20260806 00:54:21.727952 6366 ts_tablet_manager.cc:1397] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260806 00:54:21.728425 6366 raft_consensus.cc:348] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.728566 6366 raft_consensus.cc:374] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:21.728607 6366 raft_consensus.cc:729] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0e00981ffc442da6ea52aef3e5a701, State: Initialized, Role: FOLLOWER
I20260806 00:54:21.728757 6366 consensus_queue.cc:260] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.728820 6369 tablet_bootstrap.cc:492] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1: Bootstrap starting.
I20260806 00:54:21.729090 6366 ts_tablet_manager.cc:1428] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 00:54:21.729696 6369 tablet_bootstrap.cc:654] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:21.729907 6367 tablet_bootstrap.cc:492] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f: No bootstrap required, opened a new log
I20260806 00:54:21.729998 6367 ts_tablet_manager.cc:1397] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 00:54:21.730522 6367 raft_consensus.cc:348] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.730644 6367 raft_consensus.cc:374] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:21.730684 6367 raft_consensus.cc:729] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a44d05a654446bab11854680548637f, State: Initialized, Role: FOLLOWER
I20260806 00:54:21.730824 6367 consensus_queue.cc:260] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.731150 6367 ts_tablet_manager.cc:1428] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:54:21.732227 6369 tablet_bootstrap.cc:492] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1: No bootstrap required, opened a new log
I20260806 00:54:21.732323 6369 ts_tablet_manager.cc:1397] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260806 00:54:21.732718 6369 raft_consensus.cc:348] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.732828 6369 raft_consensus.cc:374] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:21.732863 6369 raft_consensus.cc:729] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28a43b0c75d14bd3b4d96545e01d59c1, State: Initialized, Role: FOLLOWER
I20260806 00:54:21.732976 6369 consensus_queue.cc:260] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:21.733230 6369 ts_tablet_manager.cc:1428] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:54:22.038383 6374 raft_consensus.cc:482] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:54:22.038585 6374 raft_consensus.cc:504] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:22.039172 6374 leader_election.cc:288] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a44d05a654446bab11854680548637f (127.27.77.193:43471), 28a43b0c75d14bd3b4d96545e01d59c1 (127.27.77.194:39969)
I20260806 00:54:22.039733 5909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64eae87430e4db8b72830afe36748af" candidate_uuid: "dc0e00981ffc442da6ea52aef3e5a701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a44d05a654446bab11854680548637f" is_pre_election: true
I20260806 00:54:22.039809 6066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64eae87430e4db8b72830afe36748af" candidate_uuid: "dc0e00981ffc442da6ea52aef3e5a701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" is_pre_election: true
I20260806 00:54:22.039925 5909 raft_consensus.cc:2454] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0e00981ffc442da6ea52aef3e5a701 in term 0.
I20260806 00:54:22.039999 6066 raft_consensus.cc:2454] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0e00981ffc442da6ea52aef3e5a701 in term 0.
I20260806 00:54:22.040370 6178 leader_election.cc:302] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28a43b0c75d14bd3b4d96545e01d59c1, dc0e00981ffc442da6ea52aef3e5a701; no voters:
I20260806 00:54:22.040576 6374 raft_consensus.cc:2781] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:54:22.040647 6374 raft_consensus.cc:482] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:54:22.040693 6374 raft_consensus.cc:3037] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:22.041787 6374 raft_consensus.cc:504] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:22.042140 6374 leader_election.cc:288] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [CANDIDATE]: Term 1 election: Requested vote from peers 6a44d05a654446bab11854680548637f (127.27.77.193:43471), 28a43b0c75d14bd3b4d96545e01d59c1 (127.27.77.194:39969)
I20260806 00:54:22.042536 5909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64eae87430e4db8b72830afe36748af" candidate_uuid: "dc0e00981ffc442da6ea52aef3e5a701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a44d05a654446bab11854680548637f"
I20260806 00:54:22.042542 6066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64eae87430e4db8b72830afe36748af" candidate_uuid: "dc0e00981ffc442da6ea52aef3e5a701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a43b0c75d14bd3b4d96545e01d59c1"
I20260806 00:54:22.042711 5909 raft_consensus.cc:3037] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:22.042718 6066 raft_consensus.cc:3037] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:22.043848 5909 raft_consensus.cc:2454] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0e00981ffc442da6ea52aef3e5a701 in term 1.
I20260806 00:54:22.043850 6066 raft_consensus.cc:2454] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0e00981ffc442da6ea52aef3e5a701 in term 1.
I20260806 00:54:22.044185 6180 leader_election.cc:302] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a44d05a654446bab11854680548637f, dc0e00981ffc442da6ea52aef3e5a701; no voters:
I20260806 00:54:22.044350 6374 raft_consensus.cc:2781] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:22.044467 6374 raft_consensus.cc:686] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [term 1 LEADER]: Becoming Leader. State: Replica: dc0e00981ffc442da6ea52aef3e5a701, State: Running, Role: LEADER
I20260806 00:54:22.044628 6374 consensus_queue.cc:237] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } }
I20260806 00:54:22.046384 5697 catalog_manager.cc:5495] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc0e00981ffc442da6ea52aef3e5a701 (127.27.77.195). New cstate: current_term: 1 leader_uuid: "dc0e00981ffc442da6ea52aef3e5a701" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc0e00981ffc442da6ea52aef3e5a701" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42217 } health_report { overall_health: HEALTHY } } }
I20260806 00:54:22.111608 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:22.108028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:47861 (local address 127.27.77.194:39969)
0806 00:54:22.108220 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:22.108225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:22.108398 (+ 173us) server_negotiation.cc:407] Connection header received
0806 00:54:22.108586 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:22.108591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:22.108644 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:22.108765 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:54:22.109071 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:22.109599 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:22.110501 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:22.110721 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:22.110829 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:22.111134 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:54:22.111205 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:54:22.111285 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:22.111385 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 00:54:22.111437 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 00:54:22.111634 6491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:22.107904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:39969 (local address 127.27.77.195:47861)
0806 00:54:22.108299 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:22.108321 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:22.108454 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:22.108786 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:22.108788 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:22.108798 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:54:22.108911 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:22.108917 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:22.109716 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:22.109721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:22.110372 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:22.110381 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:22.110841 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:22.110844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:22.110933 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:22.110987 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:54:22.111301 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:54:22.111310 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 00:54:22.111373 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 00:54:22.111465 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":112,"threads_started":1}
I20260806 00:54:22.117525 5909 raft_consensus.cc:1261] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Refusing update from remote peer dc0e00981ffc442da6ea52aef3e5a701: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:22.117578 6066 raft_consensus.cc:1261] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Refusing update from remote peer dc0e00981ffc442da6ea52aef3e5a701: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:22.117969 6374 consensus_queue.cc:1035] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a44d05a654446bab11854680548637f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 43471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:22.118160 6477 consensus_queue.cc:1035] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28a43b0c75d14bd3b4d96545e01d59c1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 39969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:22.134768 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:22.131389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:47681 (local address 127.27.77.195:42217)
0806 00:54:22.131514 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:22.131541 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:22.131725 (+ 184us) server_negotiation.cc:407] Connection header received
0806 00:54:22.131846 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:22.131849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:22.131909 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:22.131991 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:54:22.132275 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:22.132769 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:22.133640 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:22.133935 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:22.134017 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:22.134290 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:54:22.134378 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:54:22.134436 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:22.134584 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 00:54:22.134645 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:54:22.134819 6497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:22.131338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:42217 (local address 127.27.77.194:47681)
0806 00:54:22.131624 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:22.131641 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:22.131757 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:22.132020 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:22.132085 (+ 65us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:22.132104 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:54:22.132202 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:22.132207 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:22.132857 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:22.132859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:22.133526 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:22.133541 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:22.134055 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:22.134061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:22.134151 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:22.134189 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:54:22.134464 (+ 275us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:54:22.134470 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 00:54:22.134551 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 00:54:22.134657 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":101,"threads_started":1}
I20260806 00:54:22.155259 5697 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:22.155449 5697 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:22.157387 5697 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 691e88581cf7441790575f44672dcc48: Sending DeleteTablet for 3 replicas of tablet f64eae87430e4db8b72830afe36748af
I20260806 00:54:22.157929 5881 tablet_service.cc:1514] Processing DeleteTablet for tablet f64eae87430e4db8b72830afe36748af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:22 UTC) from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:32980
I20260806 00:54:22.158025 6242 tablet_service.cc:1514] Processing DeleteTablet for tablet f64eae87430e4db8b72830afe36748af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:22 UTC) from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:36104
I20260806 00:54:22.158183 6444 tablet_replica.cc:330] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f: stopping tablet replica
I20260806 00:54:22.158223 6445 tablet_replica.cc:330] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701: stopping tablet replica
I20260806 00:54:22.158335 6444 raft_consensus.cc:2229] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:22.158355 6445 raft_consensus.cc:2229] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:54:22.158480 6444 raft_consensus.cc:2258] T f64eae87430e4db8b72830afe36748af P 6a44d05a654446bab11854680548637f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:22.158587 6445 raft_consensus.cc:2258] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:22.158685 6058 tablet_service.cc:1514] Processing DeleteTablet for tablet f64eae87430e4db8b72830afe36748af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:22 UTC) from {username='oryx', principal='oryx/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:58188
I20260806 00:54:22.158906 6448 tablet_replica.cc:330] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1: stopping tablet replica
I20260806 00:54:22.159081 6448 raft_consensus.cc:2229] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:22.159209 6448 raft_consensus.cc:2258] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:22.159513 6445 ts_tablet_manager.cc:1905] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:22.159689 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 5798
I20260806 00:54:22.165448 6448 ts_tablet_manager.cc:1905] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:22.165504 6445 ts_tablet_manager.cc:1918] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:54:22.165586 6445 log.cc:1196] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/f64eae87430e4db8b72830afe36748af
I20260806 00:54:22.165863 6445 ts_tablet_manager.cc:1939] T f64eae87430e4db8b72830afe36748af P dc0e00981ffc442da6ea52aef3e5a701: Deleting consensus metadata
I20260806 00:54:22.166607 5676 catalog_manager.cc:4841] TS dc0e00981ffc442da6ea52aef3e5a701 (127.27.77.195:42217): tablet f64eae87430e4db8b72830afe36748af (table test-table [id=5db0309f27154f5789656be477489b37]) successfully deleted
I20260806 00:54:22.167973 6448 ts_tablet_manager.cc:1918] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:54:22.168084 6448 log.cc:1196] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/f64eae87430e4db8b72830afe36748af
I20260806 00:54:22.168324 6448 ts_tablet_manager.cc:1939] T f64eae87430e4db8b72830afe36748af P 28a43b0c75d14bd3b4d96545e01d59c1: Deleting consensus metadata
I20260806 00:54:22.169016 5676 catalog_manager.cc:4841] TS 28a43b0c75d14bd3b4d96545e01d59c1 (127.27.77.194:39969): tablet f64eae87430e4db8b72830afe36748af (table test-table [id=5db0309f27154f5789656be477489b37]) successfully deleted
W20260806 00:54:22.172708 5678 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:43471 (error 108)
I20260806 00:54:22.173040 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 5983
I20260806 00:54:22.175786 6502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:22.175232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:43471 (local address 127.0.0.1:53074)
0806 00:54:22.175479 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:22.175638 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:43471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":84,"threads_started":1}
W20260806 00:54:22.176137 5678 catalog_manager.cc:4584] TS 6a44d05a654446bab11854680548637f (127.27.77.193:43471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f64eae87430e4db8b72830afe36748af: Network error: Client connection negotiation failed: client connection to 127.27.77.193:43471: connect: Connection refused (error 111)
I20260806 00:54:22.185115 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 6156
I20260806 00:54:22.200158 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 5655
2026-08-06T00:54:22Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3786 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:54:22 dist-test-slave-sb2z krb5kdc[6505](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:54:22 dist-test-slave-sb2z krb5kdc[6505](info): set up 2 sockets
Aug 06 00:54:22 dist-test-slave-sb2z krb5kdc[6505](info): commencing operation
krb5kdc: starting...
W20260806 00:54:23.392601 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:54:23 dist-test-slave-sb2z krb5kdc[6505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:54:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:54:23.535807 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:34537
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:39647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:34537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:23.653882 6522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:23.654201 6522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:23.654277 6522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:23.658272 6522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:23.658372 6522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:23.658407 6522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:23.658447 6522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:23.658478 6522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:54:23.663795 6522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:39647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:34537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:34537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:23.664891 6522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:23.665967 6522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:23.674742 6528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:23.674742 6529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:23.674767 6531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:23.675204 6522 server_base.cc:1034] running on GCE node
I20260806 00:54:23.675848 6522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:23.676977 6522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:23.678189 6522 hybrid_clock.cc:648] HybridClock initialized: now 1785977663678145 us; error 35 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T00:54:23Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1478 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:54:23 dist-test-slave-sb2z krb5kdc[6537](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:54:23 dist-test-slave-sb2z krb5kdc[6537](info): set up 2 sockets
Aug 06 00:54:23 dist-test-slave-sb2z krb5kdc[6537](info): commencing operation
krb5kdc: starting...
W20260806 00:54:24.883387 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:54:24 dist-test-slave-sb2z krb5kdc[6537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:54:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:54:25.020036 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38639
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:38639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:25.137686 6553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:25.138056 6553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:25.138146 6553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:25.142264 6553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:25.142369 6553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:25.142405 6553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:25.142447 6553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:25.142484 6553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:54:25.147972 6553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:38639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:25.149114 6553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:25.150520 6553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:25.159694 6559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:25.159691 6564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:25.159687 6560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:25.159811 6553 server_base.cc:1034] running on GCE node
I20260806 00:54:25.160557 6553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:25.161701 6553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:25.162876 6553 hybrid_clock.cc:648] HybridClock initialized: now 1785977665162843 us; error 45 us; skew 500 ppm
Aug 06 00:54:25 dist-test-slave-sb2z krb5kdc[6537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977665, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:25.168556 6553 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:54:25.169554 6553 webserver.cc:460] Webserver started at http://127.27.77.254:35429/ using document root <none> and password file <none>
I20260806 00:54:25.170136 6553 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:25.170198 6553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:25.170373 6553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:25.172494 6553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "0e452575bb044530a70b9c4e23fdaa74"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "c5778e25551da238fe5638ece7ba6f1a"
server_key_iv: "944edd91316fafd0d89b54b65173a9db"
server_key_version: "encryptionkey@0"
I20260806 00:54:25.173031 6553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "0e452575bb044530a70b9c4e23fdaa74"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "c5778e25551da238fe5638ece7ba6f1a"
server_key_iv: "944edd91316fafd0d89b54b65173a9db"
server_key_version: "encryptionkey@0"
I20260806 00:54:25.176791 6553 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:54:25.179314 6571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:25.180704 6553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:54:25.180850 6553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "0e452575bb044530a70b9c4e23fdaa74"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "c5778e25551da238fe5638ece7ba6f1a"
server_key_iv: "944edd91316fafd0d89b54b65173a9db"
server_key_version: "encryptionkey@0"
I20260806 00:54:25.180980 6553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:25.201637 6553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:25.204949 6553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:25.205174 6553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:25.212859 6553 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:38639
I20260806 00:54:25.212908 6634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:38639 every 8 connection(s)
I20260806 00:54:25.213851 6553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:25.216867 6637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:25.218207 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 6553
I20260806 00:54:25.218324 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:54:25.218662 27959 external_mini_cluster.cc:1428] Setting key ef5da40f7f378812d47c12c6cd904530
I20260806 00:54:25.223981 6637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74: Bootstrap starting.
I20260806 00:54:25.226994 6637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:25.228147 6637 log.cc:824] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:54:25 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:25.230428 6637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74: No bootstrap required, opened a new log
I20260806 00:54:25.233981 6640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:25.220242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48700 (local address 127.27.77.254:38639)
0806 00:54:25.220667 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:25.220682 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:25.220718 (+ 36us) server_negotiation.cc:407] Connection header received
0806 00:54:25.221355 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:25.221373 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:25.221695 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:25.221959 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:25.223014 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:25.224056 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:25.225384 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:25.225833 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:25.225948 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:25.229337 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:25.229342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:25.229355 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:25.229381 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:25.231109 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:25.231654 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:25.231659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:25.231663 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:25.231775 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:25.232063 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:25.232065 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:25.232066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:25.232448 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:25.232643 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:25.232952 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 00:54:25.233242 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":133,"threads_started":1}
I20260806 00:54:25.234527 6637 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e452575bb044530a70b9c4e23fdaa74" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38639 } }
I20260806 00:54:25.234783 6637 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:25.234864 6637 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e452575bb044530a70b9c4e23fdaa74, State: Initialized, Role: FOLLOWER
I20260806 00:54:25.235288 6637 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e452575bb044530a70b9c4e23fdaa74" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38639 } }
I20260806 00:54:25.235410 6637 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:25.235496 6637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:25.235638 6637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:25.236519 6637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e452575bb044530a70b9c4e23fdaa74" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38639 } }
I20260806 00:54:25.236883 6637 leader_election.cc:302] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e452575bb044530a70b9c4e23fdaa74; no voters:
I20260806 00:54:25.237224 6637 leader_election.cc:288] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:54:25.237350 6644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:25.237637 6644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 1 LEADER]: Becoming Leader. State: Replica: 0e452575bb044530a70b9c4e23fdaa74, State: Running, Role: LEADER
I20260806 00:54:25.238013 6644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e452575bb044530a70b9c4e23fdaa74" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38639 } }
I20260806 00:54:25.238431 6637 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:25.239763 6645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e452575bb044530a70b9c4e23fdaa74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e452575bb044530a70b9c4e23fdaa74" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38639 } } }
I20260806 00:54:25.239915 6645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:25.240339 6646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e452575bb044530a70b9c4e23fdaa74. Latest consensus state: current_term: 1 leader_uuid: "0e452575bb044530a70b9c4e23fdaa74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e452575bb044530a70b9c4e23fdaa74" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38639 } } }
I20260806 00:54:25.240443 6646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:25.240772 6656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:25.243772 6656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:25.248847 6656 catalog_manager.cc:1347] Generated new cluster ID: 623ba71247cf46aeab4b8961256ce9a4
I20260806 00:54:25.248911 6656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:25.257736 6656 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:54:25.258701 6656 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:25.264709 6656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74: Generated new TSK 0
I20260806 00:54:25.265338 6656 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:54:25.304844 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38639
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:25.454962 6667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:25.455235 6667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:25.455282 6667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:25.459452 6667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:25.459514 6667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:25.459663 6667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:54:25.465124 6667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:25.466246 6667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:25.467319 6667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:25.477206 6677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:25.477111 6683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:25.477311 6678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:25.477603 6667 server_base.cc:1034] running on GCE node
I20260806 00:54:25.478094 6667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:25.478776 6667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:25.480006 6667 hybrid_clock.cc:648] HybridClock initialized: now 1785977665479965 us; error 55 us; skew 500 ppm
Aug 06 00:54:25 dist-test-slave-sb2z krb5kdc[6537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977665, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:25.483402 6667 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:54:25.484450 6667 webserver.cc:460] Webserver started at http://127.27.77.193:44185/ using document root <none> and password file <none>
I20260806 00:54:25.485132 6667 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:25.485199 6667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:25.485430 6667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:25.487849 6667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "85c03007073f475dba5b447ac905cecd"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "f65d8805b383c3f4a51b389587764b97"
server_key_iv: "4151a9c6116c57ba5d73820d0bb631a7"
server_key_version: "encryptionkey@0"
I20260806 00:54:25.488464 6667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "85c03007073f475dba5b447ac905cecd"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "f65d8805b383c3f4a51b389587764b97"
server_key_iv: "4151a9c6116c57ba5d73820d0bb631a7"
server_key_version: "encryptionkey@0"
I20260806 00:54:25.492446 6667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:54:25.495198 6691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:25.496661 6667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:54:25.496819 6667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "85c03007073f475dba5b447ac905cecd"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "f65d8805b383c3f4a51b389587764b97"
server_key_iv: "4151a9c6116c57ba5d73820d0bb631a7"
server_key_version: "encryptionkey@0"
I20260806 00:54:25.496946 6667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:25.513300 6667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:25.516459 6667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:25.516687 6667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:25.517338 6667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:25.518240 6667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:25.518313 6667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:25.518388 6667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:25.518435 6667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:25.529249 6667 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:37249
I20260806 00:54:25.529315 6840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:37249 every 8 connection(s)
I20260806 00:54:25.530395 6667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:54:25.536175 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 6667
I20260806 00:54:25.536345 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:54:25.536751 27959 external_mini_cluster.cc:1428] Setting key dc77a22f99a9e9de8f3112bfad5c61bd
Aug 06 00:54:25 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977665, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:25.546411 6640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:25.532511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:53981 (local address 127.27.77.254:38639)
0806 00:54:25.532659 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:25.532663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:25.533586 (+ 923us) server_negotiation.cc:407] Connection header received
0806 00:54:25.534952 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:25.534955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:25.535007 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:25.535091 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:25.536543 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:25.537302 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:25.538675 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:25.538981 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:25.539101 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:25.542185 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:25.542188 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:25.542190 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:25.542217 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:25.543839 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:25.544541 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:25.544544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:25.544545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:25.544621 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:25.545098 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:25.545101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:25.545102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:25.545311 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:25.545401 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:25.546140 (+ 739us) server_negotiation.cc:299] Negotiation successful
0806 00:54:25.546265 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:54:25.547725 6843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:25.532894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:53981)
0806 00:54:25.533405 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:25.533450 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:25.534712 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:25.535302 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:25.535309 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:25.535777 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:25.536338 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:25.536359 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:25.537450 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:25.537454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:25.538541 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:25.538555 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:25.539158 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:25.539164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:25.539405 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:25.540179 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:25.540206 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:25.541966 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:25.543968 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:25.543976 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:25.543996 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:25.544428 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:25.544729 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:25.544734 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:25.544738 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:25.544959 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:25.545437 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:25.545447 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:25.545817 (+ 370us) client_negotiation.cc:769] Sending connection context
0806 00:54:25.546076 (+ 259us) client_negotiation.cc:240] Negotiation successful
0806 00:54:25.546485 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":131,"threads_started":1}
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
I20260806 00:54:25.549154 6841 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38639
I20260806 00:54:25.549450 6841 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:25.550004 6841 heartbeater.cc:511] Master 127.27.77.254:38639 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
I20260806 00:54:25.551817 6591 ts_manager.cc:194] Registered new tserver with Master: 85c03007073f475dba5b447ac905cecd (127.27.77.193:37249)
I20260806 00:54:25.553011 6591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:53981
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:54:25.570561 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38639
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:25.688010 6846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:25.688304 6846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:25.688378 6846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:25.692540 6846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:25.692625 6846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:25.692716 6846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:54:25.697923 6846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:25.699012 6846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:25.700126 6846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:25.710295 6855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:25.710309 6854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:25.710295 6857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:25.710469 6846 server_base.cc:1034] running on GCE node
I20260806 00:54:25.711143 6846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:25.711853 6846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:25.713088 6846 hybrid_clock.cc:648] HybridClock initialized: now 1785977665713010 us; error 48 us; skew 500 ppm
Aug 06 00:54:25 dist-test-slave-sb2z krb5kdc[6537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977665, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:25.716094 6846 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:54:25.717082 6846 webserver.cc:460] Webserver started at http://127.27.77.194:37583/ using document root <none> and password file <none>
I20260806 00:54:25.717700 6846 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:25.717811 6846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:25.718055 6846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:25.720274 6846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "ddb6abb16c6b4ba2a39c78d4227199a4"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "103700d672777659e620ad43bfaeec2c"
server_key_iv: "16d6907d439b13ab1c105a7fb630892d"
server_key_version: "encryptionkey@0"
I20260806 00:54:25.720882 6846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "ddb6abb16c6b4ba2a39c78d4227199a4"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "103700d672777659e620ad43bfaeec2c"
server_key_iv: "16d6907d439b13ab1c105a7fb630892d"
server_key_version: "encryptionkey@0"
I20260806 00:54:25.724716 6846 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 00:54:25.727237 6868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:25.728543 6846 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:54:25.728700 6846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "ddb6abb16c6b4ba2a39c78d4227199a4"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "103700d672777659e620ad43bfaeec2c"
server_key_iv: "16d6907d439b13ab1c105a7fb630892d"
server_key_version: "encryptionkey@0"
I20260806 00:54:25.728821 6846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:25.740515 6846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:25.744083 6846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:25.744326 6846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:25.745004 6846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:25.745927 6846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:25.746007 6846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:25.746088 6846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:25.746150 6846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:25.756690 6846 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:39251
I20260806 00:54:25.756719 7011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:39251 every 8 connection(s)
I20260806 00:54:25.757715 6846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:54:25.758594 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 6846
I20260806 00:54:25.758730 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:54:25.759033 27959 external_mini_cluster.cc:1428] Setting key 3a1d2afc585d5c73cc0a87699584c606
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
Aug 06 00:54:25 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977665, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
I20260806 00:54:25.774573 6640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:25.759843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:60355 (local address 127.27.77.254:38639)
0806 00:54:25.759971 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:25.759976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:25.763018 (+ 3042us) server_negotiation.cc:407] Connection header received
0806 00:54:25.764469 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:25.764473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:25.764530 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:25.764633 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:25.765933 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:25.766471 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:25.767669 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:25.767901 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:25.768036 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:25.770613 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:25.770616 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:25.770619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:25.770648 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:25.772139 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:25.772713 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:25.772719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:25.772721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:25.772806 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:25.773192 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:25.773197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:25.773199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:25.773404 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:25.773531 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:25.774165 (+ 634us) server_negotiation.cc:299] Negotiation successful
0806 00:54:25.774324 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 00:54:25.775301 7016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:25.760291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:60355)
0806 00:54:25.762863 (+ 2572us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:25.762905 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:25.764257 (+ 1352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:25.764801 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:25.764807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:25.765207 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:25.765783 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:25.765800 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:25.766610 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:25.766619 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:25.767483 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:25.767502 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:25.768069 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:25.768073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:25.768253 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:25.768898 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:25.768915 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:25.770430 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:25.772268 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:25.772275 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:25.772291 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:25.772571 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:25.772906 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:25.772910 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:25.772914 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:25.773077 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:25.773549 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:25.773555 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:25.773886 (+ 331us) client_negotiation.cc:769] Sending connection context
0806 00:54:25.774069 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 00:54:25.774349 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2380,"thread_start_us":135,"threads_started":1}
I20260806 00:54:25.776712 7012 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38639
I20260806 00:54:25.776983 7012 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:25.777546 7012 heartbeater.cc:511] Master 127.27.77.254:38639 requested a full tablet report, sending...
I20260806 00:54:25.778766 6591 ts_manager.cc:194] Registered new tserver with Master: ddb6abb16c6b4ba2a39c78d4227199a4 (127.27.77.194:39251)
I20260806 00:54:25.779699 6591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:60355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:54:25.795185 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38639
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:25.910723 7021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:25.911010 7021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:25.911115 7021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:25.915247 7021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:25.915309 7021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:25.915412 7021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:54:25.920718 7021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:25.921844 7021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:25.922901 7021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:25.932823 7029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:25.932861 7028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:25.932917 7021 server_base.cc:1034] running on GCE node
W20260806 00:54:25.932861 7031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:25.933467 7021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:25.934157 7021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:25.935359 7021 hybrid_clock.cc:648] HybridClock initialized: now 1785977665935325 us; error 50 us; skew 500 ppm
Aug 06 00:54:25 dist-test-slave-sb2z krb5kdc[6537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977665, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:25.938410 7021 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:54:25.939422 7021 webserver.cc:460] Webserver started at http://127.27.77.195:38515/ using document root <none> and password file <none>
I20260806 00:54:25.940060 7021 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:25.940130 7021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:25.940379 7021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:25.942497 7021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "23ca4a3324294cc28dc84f5e55d85c33"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "66c3de276615c4ddcd7dceaf3dc406aa"
server_key_iv: "316f14b7f1bdfb53edde33573d567c56"
server_key_version: "encryptionkey@0"
I20260806 00:54:25.943092 7021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "23ca4a3324294cc28dc84f5e55d85c33"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "66c3de276615c4ddcd7dceaf3dc406aa"
server_key_iv: "316f14b7f1bdfb53edde33573d567c56"
server_key_version: "encryptionkey@0"
I20260806 00:54:25.946997 7021 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:54:25.949594 7038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:25.950874 7021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:54:25.951015 7021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "23ca4a3324294cc28dc84f5e55d85c33"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "66c3de276615c4ddcd7dceaf3dc406aa"
server_key_iv: "316f14b7f1bdfb53edde33573d567c56"
server_key_version: "encryptionkey@0"
I20260806 00:54:25.951136 7021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:25.962929 7021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:25.966145 7021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:25.966359 7021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:25.966960 7021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:25.967868 7021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:25.967935 7021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:25.968001 7021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:25.968041 7021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:25.977919 7021 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:42533
I20260806 00:54:25.977947 7192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:42533 every 8 connection(s)
I20260806 00:54:25.978811 7021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:54:25.982744 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 7021
I20260806 00:54:25.982895 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:54:25.983224 27959 external_mini_cluster.cc:1428] Setting key 4ce9f40d4c3feef7e757e48517ee2c80
Aug 06 00:54:25 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977665, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:25.993168 6640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:25.980734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:41357 (local address 127.27.77.254:38639)
0806 00:54:25.980889 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:25.980895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:25.981800 (+ 905us) server_negotiation.cc:407] Connection header received
0806 00:54:25.983436 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:25.983439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:25.983486 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:25.983635 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:25.985318 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:25.985940 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:25.987052 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:25.987260 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:25.987339 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:25.989876 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:25.989879 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:25.989881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:25.989903 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:25.991221 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:25.991765 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:25.991768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:25.991769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:25.991844 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:25.992160 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:25.992162 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:25.992163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:25.992321 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:25.992397 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:25.992953 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 00:54:25.993049 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:54:25.994081 7196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:25.981192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:41357)
0806 00:54:25.981653 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:25.981695 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:25.983206 (+ 1511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:25.983949 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:25.983957 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:25.984494 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:25.985135 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:25.985150 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:25.986070 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:25.986074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:25.986940 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:25.986950 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:25.987391 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:25.987394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:25.987588 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:25.988225 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:25.988241 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:25.989722 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:25.991329 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:25.991334 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:25.991346 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:25.991677 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:25.991926 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:25.991928 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:25.991929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:25.992075 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:25.992468 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:25.992474 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:25.992721 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 00:54:25.992905 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 00:54:25.993187 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":112,"threads_started":1}
I20260806 00:54:25.995309 7194 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38639
I20260806 00:54:25.995675 7194 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:25.996313 7194 heartbeater.cc:511] Master 127.27.77.254:38639 requested a full tablet report, sending...
I20260806 00:54:25.997516 6591 ts_manager.cc:194] Registered new tserver with Master: 23ca4a3324294cc28dc84f5e55d85c33 (127.27.77.195:42533)
I20260806 00:54:25.998121 6591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:41357
I20260806 00:54:26.009656 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.193.keytab.
Entry for principal oryx/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.193.keytab.
WARNING: no policy specified for oryx/127.27.77.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.77.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.194.keytab.
Entry for principal oryx/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.194.keytab.
WARNING: no policy specified for oryx/127.27.77.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.77.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.195.keytab.
Entry for principal oryx/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.195.keytab.
I20260806 00:54:26.094908 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 6667
I20260806 00:54:26.104975 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 6846
I20260806 00:54:26.115877 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 7021
I20260806 00:54:26.128448 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 6553
I20260806 00:54:26.140064 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38639
--webserver_interface=127.27.77.254
--webserver_port=35429
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:38639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:26.253765 7205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:26.254091 7205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:26.254174 7205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:26.258231 7205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:26.258329 7205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:26.258368 7205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:26.258407 7205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:26.258442 7205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:54:26.263752 7205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:38639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=35429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:26.264827 7205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:26.265868 7205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:26.274734 7213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:26.274897 7205 server_base.cc:1034] running on GCE node
W20260806 00:54:26.274739 7211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:26.274716 7210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:26.275751 7205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:26.276782 7205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:26.277946 7205 hybrid_clock.cc:648] HybridClock initialized: now 1785977666277935 us; error 32 us; skew 500 ppm
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977666, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:26.280942 7205 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:54:26.281888 7205 webserver.cc:460] Webserver started at http://127.27.77.254:35429/ using document root <none> and password file <none>
I20260806 00:54:26.282414 7205 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:26.282487 7205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:26.286453 7205 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:54:26.288872 7223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:26.290269 7205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 00:54:26.290434 7205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "0e452575bb044530a70b9c4e23fdaa74"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "c5778e25551da238fe5638ece7ba6f1a"
server_key_iv: "944edd91316fafd0d89b54b65173a9db"
server_key_version: "encryptionkey@0"
I20260806 00:54:26.291038 7205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:26.311218 7205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:26.314821 7205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:26.315104 7205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:26.323588 7291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:38639 every 8 connection(s)
I20260806 00:54:26.323606 7205 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:38639
I20260806 00:54:26.324857 7205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:26.327504 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 7205
I20260806 00:54:26.328043 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:37249
--local_ip_for_outbound_sockets=127.27.77.193
--tserver_master_addrs=127.27.77.254:38639
--webserver_port=44185
--webserver_interface=127.27.77.193
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.77.193
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:54:26.329866 7293 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:54:26.331723 7293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74: Bootstrap starting.
I20260806 00:54:26.356756 7293 log.cc:824] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:26.365398 7293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:54:26.365850 7293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74: Bootstrap complete.
I20260806 00:54:26.371812 7293 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e452575bb044530a70b9c4e23fdaa74" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38639 } }
I20260806 00:54:26.372365 7293 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e452575bb044530a70b9c4e23fdaa74, State: Initialized, Role: FOLLOWER
I20260806 00:54:26.372931 7293 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e452575bb044530a70b9c4e23fdaa74" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38639 } }
I20260806 00:54:26.373126 7293 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:26.373202 7293 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:26.373302 7293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:54:26.375939 7293 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e452575bb044530a70b9c4e23fdaa74" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38639 } }
I20260806 00:54:26.376322 7293 leader_election.cc:302] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e452575bb044530a70b9c4e23fdaa74; no voters:
I20260806 00:54:26.376696 7293 leader_election.cc:288] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 00:54:26.376883 7305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 00:54:26.377193 7305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [term 2 LEADER]: Becoming Leader. State: Replica: 0e452575bb044530a70b9c4e23fdaa74, State: Running, Role: LEADER
I20260806 00:54:26.377542 7305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e452575bb044530a70b9c4e23fdaa74" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38639 } }
I20260806 00:54:26.378127 7293 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:26.379314 7309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0e452575bb044530a70b9c4e23fdaa74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e452575bb044530a70b9c4e23fdaa74" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38639 } } }
I20260806 00:54:26.379347 7311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e452575bb044530a70b9c4e23fdaa74. Latest consensus state: current_term: 2 leader_uuid: "0e452575bb044530a70b9c4e23fdaa74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e452575bb044530a70b9c4e23fdaa74" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38639 } } }
I20260806 00:54:26.379443 7309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:26.379473 7311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:26.380847 7314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:26.384017 7314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:26.384717 7314 catalog_manager.cc:1259] Loaded cluster ID: 623ba71247cf46aeab4b8961256ce9a4
I20260806 00:54:26.384783 7314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:26.387202 7314 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:26.387900 7314 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0e452575bb044530a70b9c4e23fdaa74: Loaded TSK: 0
I20260806 00:54:26.389108 7314 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:54:26.462126 7295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:26.462451 7295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:26.462539 7295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:26.466835 7295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:26.466941 7295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:26.467041 7295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:54:26.472782 7295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.193.keytab
--principal=oryx/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:37249
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=44185
--tserver_master_addrs=127.27.77.254:38639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:26.473819 7295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:26.474853 7295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:26.486058 7347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:26.486145 7343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:26.486058 7341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:26.486644 7295 server_base.cc:1034] running on GCE node
I20260806 00:54:26.487058 7295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:26.487800 7295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:26.489006 7295 hybrid_clock.cc:648] HybridClock initialized: now 1785977666488976 us; error 46 us; skew 500 ppm
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977666, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:26.492033 7295 init.cc:377] Logged in from keytab as oryx/127.27.77.193@KRBTEST.COM (short username oryx)
I20260806 00:54:26.492913 7295 webserver.cc:460] Webserver started at http://127.27.77.193:44185/ using document root <none> and password file <none>
I20260806 00:54:26.493510 7295 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:26.493636 7295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:26.497968 7295 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:54:26.500286 7360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:26.501741 7295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:54:26.501899 7295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "85c03007073f475dba5b447ac905cecd"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "f65d8805b383c3f4a51b389587764b97"
server_key_iv: "4151a9c6116c57ba5d73820d0bb631a7"
server_key_version: "encryptionkey@0"
I20260806 00:54:26.502458 7295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:26.522840 7295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:26.526091 7295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:26.526355 7295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:26.527033 7295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:26.527926 7295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:26.528002 7295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:26.528074 7295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:26.528127 7295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:26.538265 7295 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:37249
I20260806 00:54:26.538300 7508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:37249 every 8 connection(s)
I20260806 00:54:26.539335 7295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:54:26.546918 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 7295
I20260806 00:54:26.547439 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:39251
--local_ip_for_outbound_sockets=127.27.77.194
--tserver_master_addrs=127.27.77.254:38639
--webserver_port=37583
--webserver_interface=127.27.77.194
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.77.194
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 00:54:26.554642 7511 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:26.541737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:41747 (local address 127.27.77.254:38639)
0806 00:54:26.542448 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:26.542468 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:26.542684 (+ 216us) server_negotiation.cc:407] Connection header received
0806 00:54:26.544352 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:26.544362 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:26.544648 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:26.544972 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:26.546148 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.547510 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.548843 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.549565 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.549693 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.553851 (+ 4158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:41747: BlockingRecv error: recv got EOF from 127.27.77.193:41747 (error 108)
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":169,"threads_started":1}
I20260806 00:54:26.554652 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:26.541865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:41747)
0806 00:54:26.542502 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:26.542564 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:26.543907 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:26.545040 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:26.545058 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:26.545502 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:26.545958 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.545972 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.547708 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.547714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.548693 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.548704 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.549752 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.549759 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.549993 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.550709 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:26.550736 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:26.553637 (+ 2901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":167,"threads_started":1}
W20260806 00:54:26.555222 7509 heartbeater.cc:644] Failed to heartbeat to 127.27.77.254:38639 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.77.254:38639: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:26.561409 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:26.555767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:51155)
0806 00:54:26.555870 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:26.555886 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:26.555992 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:26.556234 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:26.556237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:26.556400 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:26.556490 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.556495 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.557247 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.557251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.557863 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.557872 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.558353 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.558356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.558454 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.558964 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:26.558976 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:26.561270 (+ 2294us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 00:54:26.561637 7511 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:26.555871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:51155 (local address 127.27.77.254:38639)
0806 00:54:26.556013 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:26.556019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:26.556035 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:54:26.556077 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:26.556079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:26.556123 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:26.556219 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:26.556609 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.557123 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.558001 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.558235 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.558348 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.561485 (+ 3137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:51155: BlockingRecv error: recv got EOF from 127.27.77.193:51155 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:26.567867 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:26.561988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:47213)
0806 00:54:26.562272 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:26.562289 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:26.562389 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:26.562670 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:26.562674 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:26.562850 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:26.562964 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.562972 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.564034 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.564037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.564607 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.564615 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.565178 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.565181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.565244 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.565773 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:26.565795 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:26.567743 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206}
W20260806 00:54:26.568027 7511 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:26.562116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:47213 (local address 127.27.77.254:38639)
0806 00:54:26.562245 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:26.562250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:26.562360 (+ 110us) server_negotiation.cc:407] Connection header received
0806 00:54:26.562501 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:26.562504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:26.562557 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:26.562652 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:26.563106 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.563916 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.564720 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.565067 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.565167 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.567881 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:47213: BlockingRecv error: recv got EOF from 127.27.77.193:47213 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 00:54:26.568077 7509 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:54:26.670400 7513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:26.670698 7513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:26.670744 7513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:26.675467 7513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:26.675592 7513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:26.675704 7513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:54:26.681501 7513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.194.keytab
--principal=oryx/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:39251
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=37583
--tserver_master_addrs=127.27.77.254:38639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:26.682590 7513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:26.683686 7513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:26.693722 7525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:26.693712 7521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:26.693712 7522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:26.694046 7513 server_base.cc:1034] running on GCE node
I20260806 00:54:26.694597 7513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:26.695346 7513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:26.696537 7513 hybrid_clock.cc:648] HybridClock initialized: now 1785977666696518 us; error 37 us; skew 500 ppm
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977666, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:26.699405 7513 init.cc:377] Logged in from keytab as oryx/127.27.77.194@KRBTEST.COM (short username oryx)
I20260806 00:54:26.700343 7513 webserver.cc:460] Webserver started at http://127.27.77.194:37583/ using document root <none> and password file <none>
I20260806 00:54:26.700938 7513 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:26.701092 7513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:26.705041 7513 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:54:26.707320 7533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:26.708763 7513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:54:26.708890 7513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "ddb6abb16c6b4ba2a39c78d4227199a4"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "103700d672777659e620ad43bfaeec2c"
server_key_iv: "16d6907d439b13ab1c105a7fb630892d"
server_key_version: "encryptionkey@0"
I20260806 00:54:26.709483 7513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:26.720470 7513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:26.723572 7513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:26.723778 7513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:26.724380 7513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:26.725227 7513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:26.725275 7513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:26.725346 7513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:26.725381 7513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:26.735049 7513 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:39251
I20260806 00:54:26.735080 7684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:39251 every 8 connection(s)
I20260806 00:54:26.736132 7513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:26.746112 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 7513
I20260806 00:54:26.746680 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:42533
--local_ip_for_outbound_sockets=127.27.77.195
--tserver_master_addrs=127.27.77.254:38639
--webserver_port=38515
--webserver_interface=127.27.77.195
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.77.195
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 00:54:26.748133 7511 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:26.738086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:34281 (local address 127.27.77.254:38639)
0806 00:54:26.738239 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:26.738242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:26.739108 (+ 866us) server_negotiation.cc:407] Connection header received
0806 00:54:26.740507 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:26.740510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:26.740565 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:26.740642 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:26.742001 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.742591 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.743815 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.744038 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.744166 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.747960 (+ 3794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:34281: BlockingRecv error: recv got EOF from 127.27.77.194:34281 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:54:26.748610 7688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:26.738447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:34281)
0806 00:54:26.738916 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:26.738972 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:26.740295 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:26.740905 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:26.740912 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:26.741308 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:26.741785 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.741799 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.742727 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.742730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.743698 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.743708 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.744223 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.744226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.744372 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.744920 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:26.744936 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:26.747768 (+ 2832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":117,"threads_started":1}
W20260806 00:54:26.749141 7686 heartbeater.cc:644] Failed to heartbeat to 127.27.77.254:38639 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.77.254:38639: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:26.755308 7688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:26.749650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:33063)
0806 00:54:26.749780 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:26.749796 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:26.749876 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:26.750141 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:26.750143 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:26.750275 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:26.750365 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.750369 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.751427 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.751430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.752094 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.752104 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.752587 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.752590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.752688 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.753259 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:26.753271 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:26.755150 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 00:54:26.755402 7511 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:26.749746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:33063 (local address 127.27.77.254:38639)
0806 00:54:26.749884 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:26.749891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:26.749911 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:54:26.749957 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:26.749961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:26.750014 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:26.750118 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:26.750482 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.751291 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.752230 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.752451 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.752571 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.755281 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:33063: BlockingRecv error: recv got EOF from 127.27.77.194:33063 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:26.761189 7688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:26.756044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:32817)
0806 00:54:26.756143 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:26.756158 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:26.756260 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:26.756523 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:26.756526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:26.756681 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:26.756749 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.756755 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.757477 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.757481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.757995 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.758003 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.758474 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.758478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.758558 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.759105 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:26.759116 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:26.761056 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 00:54:26.761466 7511 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:26.756167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:32817 (local address 127.27.77.254:38639)
0806 00:54:26.756322 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:26.756328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:26.756342 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:54:26.756375 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:26.756378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:26.756421 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:26.756507 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:26.756863 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.757365 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.758130 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.758361 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.758474 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.761206 (+ 2732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:32817: BlockingRecv error: recv got EOF from 127.27.77.194:32817 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 00:54:26.761754 7686 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:54:26.866793 7689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:26.867063 7689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:26.867106 7689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:26.871163 7689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:26.871234 7689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:26.871292 7689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:54:26.876662 7689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/krb5kdc/oryx_127.27.77.195.keytab
--principal=oryx/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:42533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=38515
--tserver_master_addrs=127.27.77.254:38639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:26.877730 7689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:26.878809 7689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:26.889317 7700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:26.889472 7698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:26.889300 7696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:26.890229 7689 server_base.cc:1034] running on GCE node
I20260806 00:54:26.890626 7689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:26.891304 7689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:26.892498 7689 hybrid_clock.cc:648] HybridClock initialized: now 1785977666892458 us; error 50 us; skew 500 ppm
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977666, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:26.895574 7689 init.cc:377] Logged in from keytab as oryx/127.27.77.195@KRBTEST.COM (short username oryx)
I20260806 00:54:26.896590 7689 webserver.cc:460] Webserver started at http://127.27.77.195:38515/ using document root <none> and password file <none>
I20260806 00:54:26.897166 7689 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:26.897284 7689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:26.901168 7689 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:54:26.903316 7714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:26.904680 7689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:54:26.904835 7689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "23ca4a3324294cc28dc84f5e55d85c33"
format_stamp: "Formatted at 2026-08-06 00:54:25 on dist-test-slave-sb2z"
server_key: "66c3de276615c4ddcd7dceaf3dc406aa"
server_key_iv: "316f14b7f1bdfb53edde33573d567c56"
server_key_version: "encryptionkey@0"
I20260806 00:54:26.905416 7689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:26.923230 7689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:26.926781 7689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:26.927021 7689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:26.927737 7689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:26.928596 7689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:26.928670 7689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:26.928746 7689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:26.928785 7689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:26.938545 7689 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:42533
I20260806 00:54:26.938570 7864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:42533 every 8 connection(s)
I20260806 00:54:26.939620 7689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:54:26.945389 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 7689
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 00:54:26.951807 7511 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:26.941603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:59907 (local address 127.27.77.254:38639)
0806 00:54:26.941773 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:26.941778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:26.943132 (+ 1354us) server_negotiation.cc:407] Connection header received
0806 00:54:26.944005 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:26.944008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:26.944056 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:26.944139 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:26.945273 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.946022 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.947573 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.947861 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.947940 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.951679 (+ 3739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:59907: BlockingRecv error: recv got EOF from 127.27.77.195:59907 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 00:54:26.952587 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:26.942038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:59907)
0806 00:54:26.942968 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:26.943012 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:26.943813 (+ 801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:26.944269 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:26.944276 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:26.944630 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:26.945107 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.945120 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.946194 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.946198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.947407 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.947418 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.948029 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.948033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.948223 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.948891 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:26.948908 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:26.951587 (+ 2679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":559,"threads_started":1}
W20260806 00:54:26.953204 7865 heartbeater.cc:644] Failed to heartbeat to 127.27.77.254:38639 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.77.254:38639: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
I20260806 00:54:26.955696 7870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:26.946469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48710 (local address 127.27.77.254:38639)
0806 00:54:26.946714 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:26.946718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:26.946732 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:54:26.946767 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:26.946770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:26.946814 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:26.946911 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:26.947742 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.948452 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.949352 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.949591 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.949700 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.951362 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:26.951367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:26.951382 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:26.951409 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:26.953427 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:26.953982 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:26.953988 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:26.953994 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:26.954122 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:26.954418 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:26.954422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:26.954424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:26.954760 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:26.954963 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:26.955278 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 00:54:26.955512 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":76,"threads_started":1}
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:26.959638 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:26.953673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:44881)
0806 00:54:26.953788 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:26.953804 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:26.953881 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:26.954151 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:26.954153 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:26.954312 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:26.954400 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.954405 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.955120 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.955123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.956622 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.956632 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.957072 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.957076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.957173 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.957652 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:26.957671 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:26.959468 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 00:54:26.959720 7511 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:26.953755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:44881 (local address 127.27.77.254:38639)
0806 00:54:26.953876 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:26.953880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:26.953893 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:54:26.953968 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:26.953971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:26.954014 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:26.954136 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:26.954513 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.954997 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.956746 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.956960 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.957069 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.959605 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:44881: BlockingRecv error: recv got EOF from 127.27.77.195:44881 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:26 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:26.964696 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:26.960299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:56843)
0806 00:54:26.960422 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:26.960435 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:26.960509 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:26.960690 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:26.960692 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:26.960814 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:26.960881 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.960885 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.961531 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.961533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.962055 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:26.962063 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.962436 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.962439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:26.962493 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.962869 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:26.962880 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:26.964613 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 00:54:26.964831 7511 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:26.960383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:56843 (local address 127.27.77.254:38639)
0806 00:54:26.960514 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:26.960517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:26.960529 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:54:26.960560 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:26.960562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:26.960598 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:26.960661 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:26.960984 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.961435 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.962154 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:26.962346 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:26.962418 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:26.964721 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:56843: BlockingRecv error: recv got EOF from 127.27.77.195:56843 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 00:54:26.964999 7865 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 00:54:27 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:27 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:27.574587 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:27.568724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:56711)
0806 00:54:27.569029 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:27.569051 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:27.569145 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:27.569529 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:27.569531 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:27.569729 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:27.569848 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:27.569854 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:27.570676 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:27.570679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:27.571304 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:27.571313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:27.571797 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:27.571800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:27.571868 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:27.572402 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:27.572413 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:27.574444 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":73,"threads_started":1}
W20260806 00:54:27.574765 7872 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:27.568822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:56711 (local address 127.27.77.254:38639)
0806 00:54:27.569262 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:27.569265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:27.569277 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:54:27.569313 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:27.569315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:27.569354 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:27.569439 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:27.569993 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:27.570552 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:27.571431 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:27.571690 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:27.571775 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:27.574614 (+ 2839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:56711: BlockingRecv error: recv got EOF from 127.27.77.193:56711 (error 108)
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":97,"threads_started":1}
Aug 06 00:54:27 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:27 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:27.768608 7873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:27.762484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:58371)
0806 00:54:27.762848 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:27.762868 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:27.763009 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:27.763429 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:27.763431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:27.763680 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:27.763781 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:27.763787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:27.764698 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:27.764701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:27.765330 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:27.765340 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:27.765880 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:27.765883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:27.765951 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:27.766466 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:27.766477 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:27.768420 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":89,"threads_started":1}
W20260806 00:54:27.768666 7872 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:27.762552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:58371 (local address 127.27.77.254:38639)
0806 00:54:27.762721 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:27.762726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:27.763037 (+ 311us) server_negotiation.cc:407] Connection header received
0806 00:54:27.763195 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:27.763201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:27.763274 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:27.763379 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:27.763929 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:27.764553 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:27.765491 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:27.765744 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:27.765846 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:27.768542 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:58371: BlockingRecv error: recv got EOF from 127.27.77.194:58371 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 00:54:27 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:27 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:27.972157 7876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:27.965737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:47927)
0806 00:54:27.966047 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:27.966066 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:27.966184 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:27.966439 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:27.966456 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:27.966654 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:27.966771 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:27.966776 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:27.967735 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:27.967738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:27.968400 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:27.968411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:27.968988 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:27.968991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:27.969063 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:27.969617 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:27.969629 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:27.971964 (+ 2335us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":117,"threads_started":1}
W20260806 00:54:27.972196 7872 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:27.965788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:47927 (local address 127.27.77.254:38639)
0806 00:54:27.965940 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:27.965945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:27.966159 (+ 214us) server_negotiation.cc:407] Connection header received
0806 00:54:27.966271 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:27.966273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:27.966321 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:27.966393 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:27.966921 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:27.967598 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:27.968543 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:27.968833 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:27.968928 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:27.972087 (+ 3159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:47927: BlockingRecv error: recv got EOF from 127.27.77.195:47927 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 00:54:28 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:28 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:28.581872 7878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:28.575473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:44797)
0806 00:54:28.575842 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:28.575856 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:28.575958 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:28.576272 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:28.576274 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:28.576472 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:28.576578 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:28.576583 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:28.577406 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:28.577409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:28.577959 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:28.577966 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:28.578628 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:28.578632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:28.578712 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:28.579361 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:28.579382 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:28.581754 (+ 2372us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":124,"threads_started":1}
W20260806 00:54:28.582108 7880 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:28.575579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:44797 (local address 127.27.77.254:38639)
0806 00:54:28.575864 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:28.575869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:28.575927 (+ 58us) server_negotiation.cc:407] Connection header received
0806 00:54:28.576093 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:28.576095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:28.576146 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:28.576236 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:28.576688 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:28.577300 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:28.578120 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:28.578385 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:28.578595 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:28.581933 (+ 3338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:44797: BlockingRecv error: recv got EOF from 127.27.77.193:44797 (error 108)
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":117,"threads_started":1}
Aug 06 00:54:28 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:28 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:28.774744 7881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:28.769470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:44875)
0806 00:54:28.769770 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:28.769783 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:28.769878 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:28.770126 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:28.770128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:28.770311 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:28.770410 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:28.770415 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:28.771148 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:28.771150 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:28.771747 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:28.771756 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:28.772200 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:28.772203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:28.772263 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:28.772746 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:28.772758 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:28.774626 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":94,"threads_started":1}
W20260806 00:54:28.774950 7880 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:28.769485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:44875 (local address 127.27.77.254:38639)
0806 00:54:28.769650 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:28.769655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:28.769847 (+ 192us) server_negotiation.cc:407] Connection header received
0806 00:54:28.769957 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:28.769960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:28.770005 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:28.770064 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:28.770516 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:28.771036 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:28.771851 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:28.772087 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:28.772179 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:28.774800 (+ 2621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:44875: BlockingRecv error: recv got EOF from 127.27.77.194:44875 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 00:54:28 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:28 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:28.979398 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:28.973094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:34095)
0806 00:54:28.973416 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:28.973435 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:28.973536 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:28.973789 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:28.973791 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:28.974005 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:28.974113 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:28.974118 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:28.974905 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:28.974908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:28.975569 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:28.975580 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:28.976198 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:28.976206 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:28.976352 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:28.977039 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:28.977056 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:28.979243 (+ 2187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":110,"threads_started":1}
W20260806 00:54:28.979626 7880 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:28.973389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:34095 (local address 127.27.77.254:38639)
0806 00:54:28.973529 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:28.973532 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:28.973545 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:54:28.973610 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:28.973612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:28.973658 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:28.973731 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:28.974240 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:28.974778 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:28.975681 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:28.976041 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:28.976178 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:28.979428 (+ 3250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:34095: BlockingRecv error: recv got EOF from 127.27.77.195:34095 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 00:54:29 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:29 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:29.588318 7883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:29.582715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:37595)
0806 00:54:29.582993 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:29.583012 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:29.583139 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:29.583456 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:29.583459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:29.583681 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:29.583782 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:29.583788 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:29.584520 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:29.584523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:29.585092 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:29.585100 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:29.585522 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:29.585525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:29.585596 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:29.586100 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:29.586112 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:29.588174 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":102,"threads_started":1}
W20260806 00:54:29.588449 7884 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:29.582737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:37595 (local address 127.27.77.254:38639)
0806 00:54:29.583017 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:29.583022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:29.583072 (+ 50us) server_negotiation.cc:407] Connection header received
0806 00:54:29.583252 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:29.583255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:29.583312 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:29.583408 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:29.583886 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:29.584419 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:29.585198 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:29.585422 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:29.585500 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:29.588315 (+ 2815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:37595: BlockingRecv error: recv got EOF from 127.27.77.193:37595 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":112,"threads_started":1}
Aug 06 00:54:29 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:29 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:29.781821 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:29.775687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:44487)
0806 00:54:29.775999 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:29.776013 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:29.776132 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:29.776388 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:29.776390 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:29.776593 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:29.776691 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:29.776696 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:29.777469 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:29.777473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:29.778049 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:29.778059 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:29.778678 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:29.778682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:29.778758 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:29.779306 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:29.779320 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:29.781619 (+ 2299us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":143,"threads_started":1}
W20260806 00:54:29.782152 7884 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:29.775774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:44487 (local address 127.27.77.254:38639)
0806 00:54:29.775963 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:29.775968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:29.776090 (+ 122us) server_negotiation.cc:407] Connection header received
0806 00:54:29.776223 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:29.776226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:29.776274 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:29.776347 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:29.776794 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:29.777358 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:29.778240 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:29.778497 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:29.778656 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:29.781903 (+ 3247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:44487: BlockingRecv error: recv got EOF from 127.27.77.194:44487 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 06 00:54:29 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:29 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:29.986400 7886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:29.980334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:50307)
0806 00:54:29.980681 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:29.980695 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:29.980798 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:29.981113 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:29.981115 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:29.981308 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:29.981410 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:29.981416 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:29.982175 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:29.982178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:29.982752 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:29.982762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:29.983189 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:29.983192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:29.983257 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:29.983814 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:29.983828 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:29.986281 (+ 2453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":155,"threads_started":1}
W20260806 00:54:29.986563 7884 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:29.980399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:50307 (local address 127.27.77.254:38639)
0806 00:54:29.980561 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:29.980565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:29.980761 (+ 196us) server_negotiation.cc:407] Connection header received
0806 00:54:29.980930 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:29.980932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:29.980984 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:29.981058 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:29.981518 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:29.982052 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:29.982866 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:29.983080 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:29.983163 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:29.986428 (+ 3265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:50307: BlockingRecv error: recv got EOF from 127.27.77.195:50307 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 00:54:30 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:30 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:30.597785 7887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:30.589132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:39225)
0806 00:54:30.589425 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:30.589439 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:30.589531 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:30.589784 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:30.589786 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:30.589968 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:30.590066 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:30.590071 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:30.590812 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:30.590814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:30.591379 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:30.591387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:30.591826 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:30.591829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:30.591890 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:30.595611 (+ 3721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:30.595623 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:30.597634 (+ 2011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":90,"threads_started":1}
W20260806 00:54:30.597986 7889 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:30.589216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:39225 (local address 127.27.77.254:38639)
0806 00:54:30.589497 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:30.589500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:30.589514 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:54:30.589617 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:30.589620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:30.589667 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:30.589746 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:30.590172 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:30.590714 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:30.591478 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:30.591730 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:30.591807 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:30.597812 (+ 6005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:39225: BlockingRecv error: recv got EOF from 127.27.77.193:39225 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":65,"threads_started":1}
Aug 06 00:54:30 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:30 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:30.788537 7893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:30.782645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:49861)
0806 00:54:30.782989 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:30.783003 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:30.783118 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:30.783357 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:30.783360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:30.783632 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:30.783739 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:30.783745 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:30.784532 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:30.784535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:30.785105 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:30.785113 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:30.785585 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:30.785588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:30.785653 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:30.786200 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:30.786212 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:30.788351 (+ 2139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":111,"threads_started":1}
W20260806 00:54:30.788601 7889 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:30.782733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:49861 (local address 127.27.77.254:38639)
0806 00:54:30.782899 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:30.782904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:30.783084 (+ 180us) server_negotiation.cc:407] Connection header received
0806 00:54:30.783192 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:30.783194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:30.783244 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:30.783320 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:30.783855 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:30.784432 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:30.785214 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:30.785459 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:30.785544 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:30.788492 (+ 2948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:49861: BlockingRecv error: recv got EOF from 127.27.77.194:49861 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 00:54:30 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:30 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:30.993067 7894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:30.987267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:42539)
0806 00:54:30.987606 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:30.987619 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:30.987723 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:30.987969 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:30.987971 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:30.988201 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:30.988302 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:30.988307 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:30.989063 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:30.989067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:30.989637 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:30.989646 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:30.990142 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:30.990144 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:30.990215 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:30.990814 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:30.990834 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:30.992888 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":92,"threads_started":1}
W20260806 00:54:30.993115 7889 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:30.987361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:42539 (local address 127.27.77.254:38639)
0806 00:54:30.987497 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:30.987501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:30.987686 (+ 185us) server_negotiation.cc:407] Connection header received
0806 00:54:30.987802 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:30.987805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:30.987855 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:30.987928 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:30.988403 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:30.988934 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:30.989759 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:30.990001 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:30.990155 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:30.993009 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:42539: BlockingRecv error: recv got EOF from 127.27.77.195:42539 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 00:54:31 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:31 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:31.604821 7895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:31.598619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:47301)
0806 00:54:31.598937 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:31.598951 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:31.599046 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:31.599419 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:31.599422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:31.599740 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:31.599851 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:31.599858 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:31.600716 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:31.600719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:31.601315 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:31.601324 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:31.601786 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:31.601789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:31.601853 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:31.602412 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:31.602426 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:31.604622 (+ 2196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":93,"threads_started":1}
W20260806 00:54:31.604903 7897 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:31.598758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:47301 (local address 127.27.77.254:38639)
0806 00:54:31.599027 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:31.599034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:31.599052 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:54:31.599169 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:31.599173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:31.599230 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:31.599330 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:31.600004 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:31.600589 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:31.601446 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:31.601677 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:31.601761 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:31.604763 (+ 3002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:47301: BlockingRecv error: recv got EOF from 127.27.77.193:47301 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":98,"threads_started":1}
Aug 06 00:54:31 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:31 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:31.796152 7898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:31.789617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:38339)
0806 00:54:31.790006 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:31.790025 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:31.790158 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:31.790516 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:31.790520 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:31.790766 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:31.790869 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:31.790876 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:31.791852 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:31.791858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:31.792593 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:31.792603 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:31.793146 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:31.793150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:31.793247 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:31.793818 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:31.793831 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:31.795964 (+ 2133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":118,"threads_started":1}
W20260806 00:54:31.796260 7897 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:31.789816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:38339 (local address 127.27.77.254:38639)
0806 00:54:31.790009 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:31.790017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:31.790096 (+ 79us) server_negotiation.cc:407] Connection header received
0806 00:54:31.790289 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:31.790293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:31.790357 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:31.790464 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:31.791047 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:31.791729 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:31.792745 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:31.793007 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:31.793129 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:31.796145 (+ 3016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:38339: BlockingRecv error: recv got EOF from 127.27.77.194:38339 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 00:54:31 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:31 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:31.999955 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:31.994032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:40763)
0806 00:54:31.994320 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:31.994334 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:31.994433 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:31.994648 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:31.994651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:31.994847 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:31.994945 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:31.994950 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:31.995874 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:31.995879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:31.996443 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:31.996451 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:31.997014 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:31.997018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:31.997111 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:31.997724 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:31.997766 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:31.999838 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":109,"threads_started":1}
W20260806 00:54:32.000151 7897 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:31.994059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:40763 (local address 127.27.77.254:38639)
0806 00:54:31.994217 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:31.994222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:31.994448 (+ 226us) server_negotiation.cc:407] Connection header received
0806 00:54:31.994504 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:31.994506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:31.994561 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:31.994628 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:31.995070 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:31.995758 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:31.996571 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:31.996890 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:31.997008 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:31.999961 (+ 2953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:40763: BlockingRecv error: recv got EOF from 127.27.77.195:40763 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 00:54:32 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:32 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:32.612098 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:32.605741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:58757)
0806 00:54:32.606085 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:32.606101 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:32.606237 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:32.606561 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:32.606563 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:32.606757 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:32.606854 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:32.606859 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:32.607804 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:32.607808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:32.608457 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:32.608467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:32.608947 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:32.608950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:32.609019 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:32.609578 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:32.609589 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:32.611914 (+ 2325us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":73,"threads_started":1}
W20260806 00:54:32.612262 7902 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:32.605874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:58757 (local address 127.27.77.254:38639)
0806 00:54:32.606217 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:32.606220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:32.606235 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:54:32.606355 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:32.606357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:32.606401 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:32.606485 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:32.606968 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:32.607689 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:32.608573 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:32.608833 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:32.608923 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:32.612114 (+ 3191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:58757: BlockingRecv error: recv got EOF from 127.27.77.193:58757 (error 108)
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":87,"threads_started":1}
Aug 06 00:54:32 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:32 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:32.802675 7903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:32.797082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:58301)
0806 00:54:32.797366 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:32.797379 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:32.797479 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:32.797721 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:32.797723 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:32.797916 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:32.798015 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:32.798020 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:32.798800 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:32.798802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:32.799356 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:32.799365 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:32.799948 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:32.799951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:32.800017 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:32.800565 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:32.800579 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:32.802555 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":101,"threads_started":1}
W20260806 00:54:32.802870 7902 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:32.797202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:58301 (local address 127.27.77.254:38639)
0806 00:54:32.797350 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:32.797354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:32.797448 (+ 94us) server_negotiation.cc:407] Connection header received
0806 00:54:32.797556 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:32.797558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:32.797604 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:32.797681 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:32.798156 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:32.798696 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:32.799482 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:32.799832 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:32.799924 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:32.802704 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:58301: BlockingRecv error: recv got EOF from 127.27.77.194:58301 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 00:54:33 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:33 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:33.006964 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:33.000905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:32897)
0806 00:54:33.001300 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:33.001315 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:33.001423 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:33.001775 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:33.001778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:33.001985 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:33.002090 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:33.002096 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:33.002831 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:33.002834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:33.003474 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:33.003484 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:33.004026 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:33.004028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:33.004102 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:33.004722 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:33.004735 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:33.006833 (+ 2098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":131,"threads_started":1}
W20260806 00:54:33.007323 7902 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:33.001031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:32897 (local address 127.27.77.254:38639)
0806 00:54:33.001264 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:33.001270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:33.001400 (+ 130us) server_negotiation.cc:407] Connection header received
0806 00:54:33.001591 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:33.001594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:33.001648 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:33.001726 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:33.002203 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:33.002724 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:33.003654 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:33.003904 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:33.004020 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:33.007115 (+ 3095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:32897: BlockingRecv error: recv got EOF from 127.27.77.195:32897 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 06 00:54:33 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:33 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:33.620119 7905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:33.613029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:36579)
0806 00:54:33.613423 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:33.613438 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:33.613583 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:33.613891 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:33.613893 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:33.614101 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:33.614201 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:33.614206 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:33.615472 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:33.615475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:33.616104 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:33.616113 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:33.616822 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:33.616825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:33.616889 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:33.617566 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:33.617585 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:33.620001 (+ 2416us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":91,"threads_started":1}
W20260806 00:54:33.620591 7906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:33.613142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:36579 (local address 127.27.77.254:38639)
0806 00:54:33.613564 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:33.613571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:33.613590 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:54:33.613673 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:33.613678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:33.613747 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:33.613868 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:33.614328 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:33.615354 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:33.616219 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:33.616683 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:33.616815 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:33.620209 (+ 3394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:36579: BlockingRecv error: recv got EOF from 127.27.77.193:36579 (error 108)
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":80,"threads_started":1}
Aug 06 00:54:33 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:33 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:33.810431 7907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:33.803507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:42417)
0806 00:54:33.803974 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:33.803992 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:33.804112 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:33.804396 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:33.804398 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:33.804626 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:33.804735 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:33.804741 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:33.805589 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:33.805594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:33.806343 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:33.806355 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:33.806870 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:33.806874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:33.806989 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:33.807813 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:33.807829 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:33.810232 (+ 2403us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":109,"threads_started":1}
W20260806 00:54:33.810514 7906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:33.803641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:42417 (local address 127.27.77.254:38639)
0806 00:54:33.803815 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:33.803820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:33.804098 (+ 278us) server_negotiation.cc:407] Connection header received
0806 00:54:33.804192 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:33.804194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:33.804268 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:33.804338 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:33.804870 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:33.805416 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:33.806478 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:33.806709 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:33.806899 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:33.810391 (+ 3492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:42417: BlockingRecv error: recv got EOF from 127.27.77.194:42417 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 00:54:34 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:34 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:34.014268 7908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:34.007953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:56929)
0806 00:54:34.008294 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:34.008309 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:34.008412 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:34.008704 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:34.008712 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:34.008904 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:34.009069 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:34.009078 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:34.009940 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:34.009943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:34.010562 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:34.010573 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:34.011115 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:34.011118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:34.011215 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:34.011896 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:34.011912 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:34.014146 (+ 2234us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":103,"threads_started":1}
W20260806 00:54:34.014532 7906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:34.008048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:56929 (local address 127.27.77.254:38639)
0806 00:54:34.008194 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:34.008197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:34.008378 (+ 181us) server_negotiation.cc:407] Connection header received
0806 00:54:34.008512 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:34.008515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:34.008565 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:34.008643 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:34.009211 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:34.009825 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:34.010690 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:34.010972 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:34.011089 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:34.014284 (+ 3195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:56929: BlockingRecv error: recv got EOF from 127.27.77.195:56929 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 00:54:34 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:34 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:34.626794 7909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:34.620980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:51767)
0806 00:54:34.621313 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:34.621327 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:34.621418 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:34.621691 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:34.621693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:34.621880 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:34.621979 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:34.621984 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:34.622724 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:34.622726 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:34.623297 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:34.623304 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:34.623921 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:34.623926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:34.624023 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:34.624620 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:34.624632 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:34.626673 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":142,"threads_started":1}
W20260806 00:54:34.626953 7910 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:34.621189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:51767 (local address 127.27.77.254:38639)
0806 00:54:34.621459 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:34.621463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:34.621475 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:54:34.621513 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:34.621515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:34.621557 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:34.621653 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:34.622080 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:34.622628 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:34.623430 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:34.623791 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:34.623903 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:34.626814 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:51767: BlockingRecv error: recv got EOF from 127.27.77.193:51767 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":116,"threads_started":1}
Aug 06 00:54:34 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:34 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:34.818540 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:34.811402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:33191)
0806 00:54:34.811851 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:34.811865 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:34.811964 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:34.812303 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:34.812306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:34.812573 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:34.812708 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:34.812737 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:34.814247 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:34.814251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:34.815010 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:34.815022 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:34.815737 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:34.815742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:34.815828 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:34.816407 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:34.816420 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:34.818386 (+ 1966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":106,"threads_started":1}
W20260806 00:54:34.818630 7910 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:34.811478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:33191 (local address 127.27.77.254:38639)
0806 00:54:34.811666 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:34.811672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:34.811924 (+ 252us) server_negotiation.cc:407] Connection header received
0806 00:54:34.812040 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:34.812044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:34.812124 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:34.812213 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:34.812865 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:34.814054 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:34.815169 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:34.815455 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:34.815590 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:34.818526 (+ 2936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:33191: BlockingRecv error: recv got EOF from 127.27.77.194:33191 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 00:54:35 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:35 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:35.020709 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:35.015160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:41493)
0806 00:54:35.015499 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:35.015512 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:35.015676 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:35.015947 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:35.015949 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:35.016200 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:35.016337 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:35.016344 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:35.017092 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:35.017095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:35.017665 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:35.017672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:35.018149 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:35.018153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:35.018224 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:35.018730 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:35.018743 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:35.020591 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":90,"threads_started":1}
W20260806 00:54:35.020856 7910 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:35.015190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:41493 (local address 127.27.77.254:38639)
0806 00:54:35.015320 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:35.015324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:35.015649 (+ 325us) server_negotiation.cc:407] Connection header received
0806 00:54:35.015756 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:35.015759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:35.015817 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:35.015913 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:35.016458 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:35.016994 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:35.017776 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:35.017996 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:35.018074 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:35.020735 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:41493: BlockingRecv error: recv got EOF from 127.27.77.195:41493 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 00:54:35 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:35 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:35.638005 7913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:35.627766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:57845)
0806 00:54:35.628141 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:35.628160 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:35.628311 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:35.628661 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:35.628664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:35.628918 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:35.629075 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:35.629083 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:35.630025 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:35.630028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:35.630672 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:35.630683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:35.631138 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:35.631142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:35.631213 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:35.635883 (+ 4670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:35.635904 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:35.637873 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":92,"threads_started":1}
W20260806 00:54:35.638257 7914 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:35.627844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:57845 (local address 127.27.77.254:38639)
0806 00:54:35.628178 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:35.628184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:35.628224 (+ 40us) server_negotiation.cc:407] Connection header received
0806 00:54:35.628438 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:35.628441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:35.628498 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:35.628602 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:35.629206 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:35.629893 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:35.630799 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:35.631022 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:35.631110 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:35.638095 (+ 6985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:57845: BlockingRecv error: recv got EOF from 127.27.77.193:57845 (error 108)
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":118,"threads_started":1}
Aug 06 00:54:35 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:35 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:35.824851 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:35.819356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:51141)
0806 00:54:35.819672 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:35.819688 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:35.819796 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:35.820022 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:35.820024 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:35.820216 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:35.820317 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:35.820323 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:35.821058 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:35.821060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:35.821639 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:35.821647 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:35.822169 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:35.822172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:35.822236 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:35.822706 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:35.822717 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:35.824718 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":104,"threads_started":1}
W20260806 00:54:35.825094 7914 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:35.819458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:51141 (local address 127.27.77.254:38639)
0806 00:54:35.819677 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:35.819681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:35.819761 (+ 80us) server_negotiation.cc:407] Connection header received
0806 00:54:35.819868 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:35.819871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:35.819915 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:35.819986 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:35.820416 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:35.820963 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:35.821773 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:35.822015 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:35.822109 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:35.824946 (+ 2837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:51141: BlockingRecv error: recv got EOF from 127.27.77.194:51141 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 06 00:54:36 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:36 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:36.027725 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:36.021650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:49773)
0806 00:54:36.021981 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:36.021994 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:36.022140 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:36.022419 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:36.022422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:36.022638 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:36.022735 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:36.022741 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:36.023512 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:36.023517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:36.024220 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:36.024231 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:36.024704 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:36.024708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:36.024796 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:36.025361 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:36.025375 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:36.027502 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":81,"threads_started":1}
W20260806 00:54:36.027791 7914 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:36.021734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:49773 (local address 127.27.77.254:38639)
0806 00:54:36.021887 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:36.021893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:36.022070 (+ 177us) server_negotiation.cc:407] Connection header received
0806 00:54:36.022241 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:36.022245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:36.022300 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:36.022389 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:36.022854 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:36.023396 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:36.024347 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:36.024574 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:36.024686 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:36.027691 (+ 3005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:49773: BlockingRecv error: recv got EOF from 127.27.77.195:49773 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 00:54:36 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:36 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:36.644794 7918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:36.638815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:41819)
0806 00:54:36.639126 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:36.639141 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:36.639236 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:36.639500 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:36.639508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:36.639854 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:36.639959 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:36.639965 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:36.640733 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:36.640736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:36.641328 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:36.641338 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:36.641783 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:36.641785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:36.641848 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:36.642439 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:36.642458 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:36.644646 (+ 2188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":110,"threads_started":1}
W20260806 00:54:36.644958 7919 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:36.638971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:41819 (local address 127.27.77.254:38639)
0806 00:54:36.639206 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:36.639210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:36.639224 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:54:36.639340 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:36.639343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:36.639388 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:36.639466 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:36.640070 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:36.640627 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:36.641443 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:36.641683 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:36.641764 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:36.644839 (+ 3075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:41819: BlockingRecv error: recv got EOF from 127.27.77.193:41819 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":69,"threads_started":1}
Aug 06 00:54:36 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:36 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:36.831998 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:36.825675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:39043)
0806 00:54:36.826095 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:36.826109 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:36.826249 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:36.826566 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:36.826570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:36.826793 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:36.826898 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:36.826904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:36.827819 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:36.827824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:36.828459 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:36.828468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:36.828996 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:36.829000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:36.829099 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:36.829647 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:36.829660 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:36.831835 (+ 2175us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":135,"threads_started":1}
W20260806 00:54:36.832197 7919 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:36.825786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:39043 (local address 127.27.77.254:38639)
0806 00:54:36.825970 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:36.825976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:36.826186 (+ 210us) server_negotiation.cc:407] Connection header received
0806 00:54:36.826359 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:36.826363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:36.826420 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:36.826515 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:36.827025 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:36.827699 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:36.828598 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:36.828865 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:36.828987 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:36.832008 (+ 3021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:39043: BlockingRecv error: recv got EOF from 127.27.77.194:39043 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 00:54:37 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:37 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:37.034514 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:37.028720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:56591)
0806 00:54:37.029018 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:37.029032 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:37.029127 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:37.029368 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:37.029370 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:37.029561 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:37.029673 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:37.029678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:37.030432 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:37.030435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:37.030991 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:37.030999 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:37.031477 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:37.031480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:37.031609 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:37.032168 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:37.032180 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:37.034333 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":107,"threads_started":1}
W20260806 00:54:37.034564 7919 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:37.028887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:56591 (local address 127.27.77.254:38639)
0806 00:54:37.029059 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:37.029062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:37.029081 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:54:37.029201 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:37.029203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:37.029248 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:37.029331 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:37.029790 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:37.030322 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:37.031137 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:37.031374 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:37.031457 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:37.034457 (+ 3000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:56591: BlockingRecv error: recv got EOF from 127.27.77.195:56591 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 06 00:54:37 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:37 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:37.651290 7926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:37.645736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:53455)
0806 00:54:37.646016 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:37.646029 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:37.646165 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:37.646421 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:37.646423 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:37.646611 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:37.646711 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:37.646716 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:37.647516 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:37.647518 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:37.648148 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:37.648158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:37.648620 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:37.648623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:37.648689 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:37.649273 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:37.649285 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:37.651169 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":92,"threads_started":1}
W20260806 00:54:37.651470 7929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:37.645807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:53455 (local address 127.27.77.254:38639)
0806 00:54:37.646069 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:37.646074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:37.646121 (+ 47us) server_negotiation.cc:407] Connection header received
0806 00:54:37.646245 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:37.646247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:37.646293 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:37.646385 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:37.646825 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:37.647403 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:37.648279 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:37.648510 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:37.648597 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:37.651326 (+ 2729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:53455: BlockingRecv error: recv got EOF from 127.27.77.193:53455 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":102,"threads_started":1}
Aug 06 00:54:37 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:37 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:37.838552 7931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:37.832918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:57523)
0806 00:54:37.833407 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:37.833420 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:37.833528 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:37.833753 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:37.833755 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:37.833936 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:37.834078 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:37.834083 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:37.834800 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:37.834802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:37.835406 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:37.835416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:37.835836 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:37.835839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:37.835907 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:37.836469 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:37.836481 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:37.838427 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":108,"threads_started":1}
W20260806 00:54:37.838738 7929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:37.833025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:57523 (local address 127.27.77.254:38639)
0806 00:54:37.833229 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:37.833234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:37.833493 (+ 259us) server_negotiation.cc:407] Connection header received
0806 00:54:37.833598 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:37.833601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:37.833649 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:37.833718 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:37.834182 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:37.834700 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:37.835506 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:37.835738 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:37.835812 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:37.838577 (+ 2765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:57523: BlockingRecv error: recv got EOF from 127.27.77.194:57523 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 00:54:38 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:38 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:38.041744 7932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:38.035442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:50917)
0806 00:54:38.035804 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:38.035821 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:38.035948 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:38.036270 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:38.036272 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:38.036520 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:38.036623 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:38.036629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:38.037407 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:38.037413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:38.038016 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:38.038025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:38.038673 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:38.038679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:38.038788 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:38.039453 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:38.039465 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:38.041621 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":111,"threads_started":1}
W20260806 00:54:38.041882 7929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:38.035613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:50917 (local address 127.27.77.254:38639)
0806 00:54:38.035789 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:38.035794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:38.035889 (+ 95us) server_negotiation.cc:407] Connection header received
0806 00:54:38.036097 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:38.036099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:38.036146 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:38.036246 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:38.036753 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:38.037292 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:38.038143 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:38.038381 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:38.038507 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:38.041757 (+ 3250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:50917: BlockingRecv error: recv got EOF from 127.27.77.195:50917 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 00:54:38 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:38 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:38.657936 7933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:38.652246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:44923)
0806 00:54:38.652570 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:38.652584 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:38.652679 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:38.652917 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:38.652919 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:38.653113 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:38.653220 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:38.653225 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:38.654077 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:38.654080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:38.654635 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:38.654644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:38.655140 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:38.655143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:38.655212 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:38.655850 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:38.655862 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:38.657813 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":106,"threads_started":1}
W20260806 00:54:38.658154 7934 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:38.652313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:44923 (local address 127.27.77.254:38639)
0806 00:54:38.652634 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:38.652638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:38.652652 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:54:38.652752 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:38.652755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:38.652799 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:38.652884 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:38.653351 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:38.653949 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:38.654768 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:38.655026 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:38.655124 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:38.657991 (+ 2867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:44923: BlockingRecv error: recv got EOF from 127.27.77.193:44923 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":105,"threads_started":1}
Aug 06 00:54:38 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:38 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:38.845245 7936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:38.839407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:38767)
0806 00:54:38.839705 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:38.839724 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:38.839855 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:38.840157 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:38.840159 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:38.840362 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:38.840457 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:38.840463 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:38.841246 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:38.841251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:38.841807 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:38.841815 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:38.842280 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:38.842283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:38.842368 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:38.842926 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:38.842938 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:38.845064 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":127,"threads_started":1}
W20260806 00:54:38.845310 7934 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:38.839471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:38767 (local address 127.27.77.254:38639)
0806 00:54:38.839676 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:38.839681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:38.839793 (+ 112us) server_negotiation.cc:407] Connection header received
0806 00:54:38.839951 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:38.839955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:38.840011 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:38.840095 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:38.840573 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:38.841133 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:38.841924 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:38.842167 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:38.842272 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:38.845195 (+ 2923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:38767: BlockingRecv error: recv got EOF from 127.27.77.194:38767 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 00:54:39 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:39 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:39.049203 7937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:39.042635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:43185)
0806 00:54:39.042988 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:39.043004 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:39.043153 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:39.043451 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:39.043455 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:39.043693 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:39.043797 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:39.043802 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:39.044603 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:39.044609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:39.045171 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:39.045179 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:39.045640 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:39.045644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:39.045731 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:39.046446 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:39.046459 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:39.049005 (+ 2546us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":98,"threads_started":1}
W20260806 00:54:39.049275 7934 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:39.042785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:43185 (local address 127.27.77.254:38639)
0806 00:54:39.042964 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:39.042969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:39.043088 (+ 119us) server_negotiation.cc:407] Connection header received
0806 00:54:39.043257 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:39.043260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:39.043331 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:39.043418 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:39.043920 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:39.044485 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:39.045291 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:39.045524 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:39.045633 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:39.049142 (+ 3509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:43185: BlockingRecv error: recv got EOF from 127.27.77.195:43185 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 00:54:39 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:39 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:39.665380 7938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:39.658919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:32877)
0806 00:54:39.659263 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:39.659280 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:39.659381 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:39.659818 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:39.659822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:39.660113 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:39.660227 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:39.660235 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:39.661326 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:39.661329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:39.662035 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:39.662047 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:39.662581 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:39.662584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:39.662652 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:39.663289 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:39.663335 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:39.665255 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":132,"threads_started":1}
W20260806 00:54:39.665627 7940 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:39.659082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:32877 (local address 127.27.77.254:38639)
0806 00:54:39.659339 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:39.659343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:39.659359 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:54:39.659547 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:39.659551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:39.659605 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:39.659708 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:39.660354 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:39.661175 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:39.662193 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:39.662450 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:39.662561 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:39.665446 (+ 2885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:32877: BlockingRecv error: recv got EOF from 127.27.77.193:32877 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":97,"threads_started":1}
Aug 06 00:54:39 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:39 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:39.852257 7941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:39.846177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:45725)
0806 00:54:39.846449 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:39.846463 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:39.846565 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:39.846877 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:39.846879 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:39.847071 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:39.847173 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:39.847178 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:39.848122 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:39.848125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:39.848713 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:39.848722 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:39.849250 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:39.849254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:39.849325 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:39.849924 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:39.849937 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:39.852121 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":95,"threads_started":1}
W20260806 00:54:39.852326 7940 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:39.846332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:45725 (local address 127.27.77.254:38639)
0806 00:54:39.846481 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:39.846484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:39.846514 (+ 30us) server_negotiation.cc:407] Connection header received
0806 00:54:39.846699 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:39.846702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:39.846757 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:39.846842 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:39.847311 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:39.847988 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:39.848838 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:39.849082 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:39.849178 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:39.852232 (+ 3054us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:45725: BlockingRecv error: recv got EOF from 127.27.77.194:45725 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 00:54:40 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:40 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:40.055778 7942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:40.050120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:45129)
0806 00:54:40.050470 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:40.050487 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:40.050612 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:40.050857 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:40.050859 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:40.051148 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:40.051258 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:40.051264 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:40.052095 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:40.052098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:40.052678 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:40.052687 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:40.053098 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:40.053101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:40.053163 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:40.053657 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:40.053668 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:40.055649 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":131,"threads_started":1}
W20260806 00:54:40.055908 7940 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:40.050121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:45129 (local address 127.27.77.254:38639)
0806 00:54:40.050257 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:40.050262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:40.050575 (+ 313us) server_negotiation.cc:407] Connection header received
0806 00:54:40.050712 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:40.050715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:40.050774 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:40.050840 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:40.051359 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:40.051959 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:40.052785 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:40.053002 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:40.053111 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:40.055790 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:45129: BlockingRecv error: recv got EOF from 127.27.77.195:45129 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 00:54:40 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:40 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:40.677039 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:40.666257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:36789)
0806 00:54:40.666574 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:40.666605 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:40.666747 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:40.667073 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:40.667076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:40.667338 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:40.667473 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:40.667482 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:40.668476 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:40.668479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:40.669056 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:40.669064 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:40.669530 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:40.669532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:40.669612 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:40.674925 (+ 5313us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:40.674939 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:40.676900 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":96,"threads_started":1}
W20260806 00:54:40.677277 7944 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:40.666388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:36789 (local address 127.27.77.254:38639)
0806 00:54:40.666742 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:40.666745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:40.666760 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:54:40.666866 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:40.666869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:40.666934 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:40.667029 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:40.667642 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:40.668364 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:40.669187 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:40.669421 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:40.669512 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:40.677108 (+ 7596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:36789: BlockingRecv error: recv got EOF from 127.27.77.193:36789 (error 108)
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":95,"threads_started":1}
Aug 06 00:54:40 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:40 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:40.859050 7945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:40.853146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:59279)
0806 00:54:40.853431 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:40.853444 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:40.853555 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:40.853850 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:40.853852 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:40.854090 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:40.854196 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:40.854202 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:40.854973 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:40.854976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:40.855737 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:40.855747 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:40.856303 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:40.856307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:40.856372 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:40.856992 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:40.857007 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:40.858913 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":99,"threads_started":1}
W20260806 00:54:40.859210 7944 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:40.853255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:59279 (local address 127.27.77.254:38639)
0806 00:54:40.853406 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:40.853409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:40.853514 (+ 105us) server_negotiation.cc:407] Connection header received
0806 00:54:40.853635 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:40.853653 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:40.853700 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:40.853779 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:40.854313 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:40.854860 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:40.855855 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:40.856157 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:40.856244 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:40.859059 (+ 2815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:59279: BlockingRecv error: recv got EOF from 127.27.77.194:59279 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 00:54:41 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:41 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.195@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:41.062944 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:41.056702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.195:56769)
0806 00:54:41.057102 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:41.057117 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:41.057279 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:41.057563 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:41.057566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:41.057797 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:41.057898 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:41.057904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:41.058717 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:41.058722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:41.059327 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:41.059335 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:41.059877 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:41.059880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:41.059973 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:41.060611 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:41.060630 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:41.062814 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":91,"threads_started":1}
W20260806 00:54:41.063120 7944 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:41.056743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:56769 (local address 127.27.77.254:38639)
0806 00:54:41.056896 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:41.056901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:41.057220 (+ 319us) server_negotiation.cc:407] Connection header received
0806 00:54:41.057390 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:41.057394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:41.057450 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:41.057534 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:41.058024 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:41.058588 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:41.059454 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:41.059757 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:41.059868 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:41.062959 (+ 3091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.195:56769: BlockingRecv error: recv got EOF from 127.27.77.195:56769 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 00:54:41 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:41 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.193@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:41.684167 7951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:41.678139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.193:51563)
0806 00:54:41.678472 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:41.678487 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:41.678581 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:41.678877 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:41.678880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:41.679071 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:41.679168 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:41.679173 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:41.680186 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:41.680190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:41.681021 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:41.681032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:41.681560 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:41.681564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:41.681630 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:41.682187 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:41.682199 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:41.684038 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":114,"threads_started":1}
W20260806 00:54:41.684314 7952 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:41.678228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:51563 (local address 127.27.77.254:38639)
0806 00:54:41.678511 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:41.678515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:41.678552 (+ 37us) server_negotiation.cc:407] Connection header received
0806 00:54:41.678672 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:41.678674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:41.678721 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:41.678796 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:41.679290 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:41.680073 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:41.681163 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:41.681440 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:41.681527 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:41.684182 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.193:51563: BlockingRecv error: recv got EOF from 127.27.77.193:51563 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":96,"threads_started":1}
Aug 06 00:54:41 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:54:41 dist-test-slave-sb2z krb5kdc[6537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.77.194@KRBTEST.COM for oryx/127.27.77.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:54:41.865752 7954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:41.859989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38639 (local address 127.27.77.194:37943)
0806 00:54:41.860281 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:41.860300 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:41.860421 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:41.860697 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:41.860700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:41.860934 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:41.861031 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:41.861038 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:41.861821 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:41.861826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:41.862389 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:41.862396 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:41.862869 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:41.862873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:41.862962 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:41.863518 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:41.863548 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:41.865542 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.77.254:38639: Server oryx/127.27.77.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":92,"threads_started":1}
W20260806 00:54:41.865775 7952 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:41.860074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:37943 (local address 127.27.77.254:38639)
0806 00:54:41.860247 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:41.860252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:41.860372 (+ 120us) server_negotiation.cc:407] Connection header received
0806 00:54:41.860522 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:41.860527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:41.860575 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:41.860675 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:41.861172 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:41.861710 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:41.862509 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:41.862751 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:41.862861 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:41.865671 (+ 2810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.77.194:37943: BlockingRecv error: recv got EOF from 127.27.77.194:37943 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:54:41.948664 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 7295
I20260806 00:54:41.960822 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 7513
I20260806 00:54:41.969600 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 7689
I20260806 00:54:41.984375 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 7205
2026-08-06T00:54:41Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18305 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T00:54:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:42Z Disabled control of system clock
I20260806 00:54:42.023588 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:36415
--webserver_interface=127.27.77.254
--webserver_port=0
--builtin_ntp_servers=127.27.77.212:34699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:36415
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 00:54:42.139465 7959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:42.139799 7959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:42.139868 7959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:42.143901 7959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:42.143980 7959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:42.144009 7959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:42.144037 7959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:42.144052 7959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:54:42.148730 7959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:34699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:36415
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:36415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:42.149717 7959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:42.151017 7959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:42.159646 7968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:42.159768 7971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:42.159828 7959 server_base.cc:1034] running on GCE node
W20260806 00:54:42.159687 7964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:42.160393 7959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:42.161485 7959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:42.162680 7959 hybrid_clock.cc:648] HybridClock initialized: now 1785977682162649 us; error 40 us; skew 500 ppm
I20260806 00:54:42.164585 7959 webserver.cc:460] Webserver started at http://127.27.77.254:38105/ using document root <none> and password file <none>
I20260806 00:54:42.165174 7959 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:42.165263 7959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:42.165498 7959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:42.167496 7959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "13a5805dad8145bd8f0ae1693b09e81e"
format_stamp: "Formatted at 2026-08-06 00:54:42 on dist-test-slave-sb2z"
server_key: "d8ff46fa45585d099efe74384b81f477"
server_key_iv: "9a3d8e5efc12d5c187eba268748f5ceb"
server_key_version: "encryptionkey@0"
I20260806 00:54:42.168120 7959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "13a5805dad8145bd8f0ae1693b09e81e"
format_stamp: "Formatted at 2026-08-06 00:54:42 on dist-test-slave-sb2z"
server_key: "d8ff46fa45585d099efe74384b81f477"
server_key_iv: "9a3d8e5efc12d5c187eba268748f5ceb"
server_key_version: "encryptionkey@0"
I20260806 00:54:42.171808 7959 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:54:42.174185 7978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:42.175451 7959 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:54:42.175657 7959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "13a5805dad8145bd8f0ae1693b09e81e"
format_stamp: "Formatted at 2026-08-06 00:54:42 on dist-test-slave-sb2z"
server_key: "d8ff46fa45585d099efe74384b81f477"
server_key_iv: "9a3d8e5efc12d5c187eba268748f5ceb"
server_key_version: "encryptionkey@0"
I20260806 00:54:42.175781 7959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:42.185271 7959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:42.185901 7959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:42.186096 7959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:42.193212 7959 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:36415
I20260806 00:54:42.193233 8051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:36415 every 8 connection(s)
I20260806 00:54:42.194111 7959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:42.197283 8052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:42.199817 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 7959
I20260806 00:54:42.199950 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:54:42.200268 27959 external_mini_cluster.cc:1428] Setting key f2d56cd06f727723b4d45e1261abde5d
I20260806 00:54:42.203336 8052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e: Bootstrap starting.
I20260806 00:54:42.205569 8052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:42.206445 8052 log.cc:824] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:42.208515 8052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e: No bootstrap required, opened a new log
I20260806 00:54:42.209560 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:42.201788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44372 (local address 127.27.77.254:36415)
0806 00:54:42.202436 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:42.202450 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:42.202487 (+ 37us) server_negotiation.cc:407] Connection header received
0806 00:54:42.203114 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:42.203132 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:42.203611 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:42.203941 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:42.204376 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:42.205315 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:42.206285 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:42.206798 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:42.206989 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:42.207858 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:42.207864 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:42.207876 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:42.208250 (+ 374us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:42.208301 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:42.208320 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:42.208432 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:42.208583 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 00:54:42.208864 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":131,"threads_started":1}
I20260806 00:54:42.212839 8052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13a5805dad8145bd8f0ae1693b09e81e" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 36415 } }
I20260806 00:54:42.213093 8052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:42.213191 8052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13a5805dad8145bd8f0ae1693b09e81e, State: Initialized, Role: FOLLOWER
I20260806 00:54:42.213634 8052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13a5805dad8145bd8f0ae1693b09e81e" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 36415 } }
I20260806 00:54:42.213796 8052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:42.213891 8052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:42.214016 8052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:42.215140 8052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13a5805dad8145bd8f0ae1693b09e81e" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 36415 } }
I20260806 00:54:42.215515 8052 leader_election.cc:302] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13a5805dad8145bd8f0ae1693b09e81e; no voters:
I20260806 00:54:42.215900 8052 leader_election.cc:288] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:54:42.216079 8058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:42.216346 8058 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [term 1 LEADER]: Becoming Leader. State: Replica: 13a5805dad8145bd8f0ae1693b09e81e, State: Running, Role: LEADER
I20260806 00:54:42.216727 8058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13a5805dad8145bd8f0ae1693b09e81e" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 36415 } }
I20260806 00:54:42.217249 8052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:42.218353 8059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13a5805dad8145bd8f0ae1693b09e81e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13a5805dad8145bd8f0ae1693b09e81e" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 36415 } } }
I20260806 00:54:42.218508 8059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:42.218864 8068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:42.219633 8060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13a5805dad8145bd8f0ae1693b09e81e. Latest consensus state: current_term: 1 leader_uuid: "13a5805dad8145bd8f0ae1693b09e81e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13a5805dad8145bd8f0ae1693b09e81e" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 36415 } } }
I20260806 00:54:42.219745 8060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:42.222383 8068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:42.228753 8068 catalog_manager.cc:1347] Generated new cluster ID: 9491ca4eb7304809ba9bee320c8a374b
I20260806 00:54:42.228834 8068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:42.244482 8068 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:54:42.245471 8068 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:42.251302 8068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13a5805dad8145bd8f0ae1693b09e81e: Generated new TSK 0
I20260806 00:54:42.251988 8068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:54:42.259614 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:36415
--builtin_ntp_servers=127.27.77.212:34699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 00:54:42.384949 8081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:42.385210 8081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:42.385247 8081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:42.389376 8081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:42.389432 8081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:42.389485 8081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:54:42.394170 8081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:34699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:36415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:42.395246 8081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:42.396808 8081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:42.406212 8089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:42.406234 8087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:42.406222 8086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:42.406471 8081 server_base.cc:1034] running on GCE node
I20260806 00:54:42.407029 8081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:42.407757 8081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:42.408926 8081 hybrid_clock.cc:648] HybridClock initialized: now 1785977682408907 us; error 34 us; skew 500 ppm
I20260806 00:54:42.410781 8081 webserver.cc:460] Webserver started at http://127.27.77.193:33439/ using document root <none> and password file <none>
I20260806 00:54:42.411392 8081 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:42.411458 8081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:42.411708 8081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:42.413760 8081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "a6b95870e18a421cb75693bc98807995"
format_stamp: "Formatted at 2026-08-06 00:54:42 on dist-test-slave-sb2z"
server_key: "f98ad54c5271ba39b769c26574922bb2"
server_key_iv: "88e1e832f7a59aa52c7dbcb9c4b4c75d"
server_key_version: "encryptionkey@0"
I20260806 00:54:42.414335 8081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "a6b95870e18a421cb75693bc98807995"
format_stamp: "Formatted at 2026-08-06 00:54:42 on dist-test-slave-sb2z"
server_key: "f98ad54c5271ba39b769c26574922bb2"
server_key_iv: "88e1e832f7a59aa52c7dbcb9c4b4c75d"
server_key_version: "encryptionkey@0"
I20260806 00:54:42.418184 8081 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:54:42.420887 8097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:42.422075 8081 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:54:42.422215 8081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "a6b95870e18a421cb75693bc98807995"
format_stamp: "Formatted at 2026-08-06 00:54:42 on dist-test-slave-sb2z"
server_key: "f98ad54c5271ba39b769c26574922bb2"
server_key_iv: "88e1e832f7a59aa52c7dbcb9c4b4c75d"
server_key_version: "encryptionkey@0"
I20260806 00:54:42.422322 8081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:42.438081 8081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:42.438771 8081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:42.438966 8081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:42.439630 8081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:42.440582 8081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:42.440636 8081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:42.440697 8081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:42.440730 8081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:42.450044 8081 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:36475
I20260806 00:54:42.450075 8261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:36475 every 8 connection(s)
I20260806 00:54:42.450945 8081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:54:42.457834 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 8081
I20260806 00:54:42.457966 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:54:42.458302 27959 external_mini_cluster.cc:1428] Setting key d3a0ff66785b90139d43e84f5eb80198
I20260806 00:54:42.460656 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:36415
--builtin_ntp_servers=127.27.77.212:34699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 00:54:42.461283 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:42.452882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:49689 (local address 127.27.77.254:36415)
0806 00:54:42.453070 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:42.453074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:42.454100 (+ 1026us) server_negotiation.cc:407] Connection header received
0806 00:54:42.455560 (+ 1460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:42.455563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:42.455622 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:42.455707 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:42.456608 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:42.457166 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:42.458857 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:42.459253 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:42.459388 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:42.460640 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:42.460644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:42.460647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:42.460661 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:42.460685 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:42.460696 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:42.460784 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:42.460990 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 00:54:42.461072 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:54:42.462105 8265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:42.453278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:36415 (local address 127.27.77.193:49689)
0806 00:54:42.453946 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:42.453986 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:42.455328 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:42.455894 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:42.455900 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:42.455991 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:42.456461 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:42.456473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:42.457306 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:42.457309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:42.458723 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:42.458739 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:42.459436 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:42.459439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:42.459671 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:42.460369 (+ 698us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:54:42.460371 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:42.460412 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:54:42.460478 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:42.460802 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:42.460815 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:42.460824 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 00:54:42.460973 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 00:54:42.461252 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":116,"threads_started":1}
I20260806 00:54:42.463441 8262 heartbeater.cc:347] Connected to a master server at 127.27.77.254:36415
I20260806 00:54:42.463709 8262 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:42.464298 8262 heartbeater.cc:511] Master 127.27.77.254:36415 requested a full tablet report, sending...
I20260806 00:54:42.465873 7998 ts_manager.cc:194] Registered new tserver with Master: a6b95870e18a421cb75693bc98807995 (127.27.77.193:36475)
I20260806 00:54:42.467011 7998 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.77.193:49689
W20260806 00:54:42.575794 8267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:42.576078 8267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:42.576145 8267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:42.580204 8267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:42.580281 8267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:42.580363 8267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:54:42.585104 8267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:34699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:36415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:42.586213 8267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:42.587656 8267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:42.597468 8274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:42.597390 8276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:42.597361 8273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:42.597931 8267 server_base.cc:1034] running on GCE node
I20260806 00:54:42.598353 8267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:42.599050 8267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:42.600222 8267 hybrid_clock.cc:648] HybridClock initialized: now 1785977682600206 us; error 27 us; skew 500 ppm
I20260806 00:54:42.602114 8267 webserver.cc:460] Webserver started at http://127.27.77.194:38857/ using document root <none> and password file <none>
I20260806 00:54:42.602782 8267 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:42.602855 8267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:42.603121 8267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:42.605396 8267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "c0374a64ee0840cabe203e97a8ea4e9c"
format_stamp: "Formatted at 2026-08-06 00:54:42 on dist-test-slave-sb2z"
server_key: "281053a9e8611f5b5e96bf9a79e5da61"
server_key_iv: "262688d95cf5a02e0ed3880eb056c204"
server_key_version: "encryptionkey@0"
I20260806 00:54:42.606094 8267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "c0374a64ee0840cabe203e97a8ea4e9c"
format_stamp: "Formatted at 2026-08-06 00:54:42 on dist-test-slave-sb2z"
server_key: "281053a9e8611f5b5e96bf9a79e5da61"
server_key_iv: "262688d95cf5a02e0ed3880eb056c204"
server_key_version: "encryptionkey@0"
I20260806 00:54:42.610194 8267 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 00:54:42.613004 8283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:42.614367 8267 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:54:42.614512 8267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "c0374a64ee0840cabe203e97a8ea4e9c"
format_stamp: "Formatted at 2026-08-06 00:54:42 on dist-test-slave-sb2z"
server_key: "281053a9e8611f5b5e96bf9a79e5da61"
server_key_iv: "262688d95cf5a02e0ed3880eb056c204"
server_key_version: "encryptionkey@0"
I20260806 00:54:42.614650 8267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:42.628538 8267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:42.629216 8267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:42.629405 8267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:42.629992 8267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:42.630935 8267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:42.630992 8267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:42.631058 8267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:42.631093 8267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:42.640084 8267 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:35737
I20260806 00:54:42.640113 8436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:35737 every 8 connection(s)
I20260806 00:54:42.641031 8267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:54:42.648713 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 8267
I20260806 00:54:42.648818 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:54:42.649169 27959 external_mini_cluster.cc:1428] Setting key 023a7983c24b357174bc95b053cff04b
I20260806 00:54:42.650395 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:42.642914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:46019 (local address 127.27.77.254:36415)
0806 00:54:42.643060 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:42.643064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:42.644115 (+ 1051us) server_negotiation.cc:407] Connection header received
0806 00:54:42.645515 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:42.645518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:42.645579 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:42.645691 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:42.646568 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:42.647109 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:42.648321 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:42.648544 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:42.648659 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:42.649892 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:42.649896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:42.649899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:42.649913 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:42.649940 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:42.649949 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:42.650027 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:42.650189 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 00:54:42.650255 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:54:42.651295 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:36415
--builtin_ntp_servers=127.27.77.212:34699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 00:54:42.651454 8439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:42.643307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:36415 (local address 127.27.77.194:46019)
0806 00:54:42.643964 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:42.644003 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:42.645309 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:42.645867 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:42.645873 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:42.645954 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:42.646428 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:42.646439 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:42.647248 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:42.647251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:42.648194 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:42.648204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:42.648712 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:42.648717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:42.648967 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:42.649616 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:54:42.649618 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:42.649656 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:54:42.649718 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:42.650029 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:42.650041 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:42.650050 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 00:54:42.650185 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 00:54:42.650454 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":132,"threads_started":1}
I20260806 00:54:42.652674 8437 heartbeater.cc:347] Connected to a master server at 127.27.77.254:36415
I20260806 00:54:42.652920 8437 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:42.653508 8437 heartbeater.cc:511] Master 127.27.77.254:36415 requested a full tablet report, sending...
I20260806 00:54:42.654542 7998 ts_manager.cc:194] Registered new tserver with Master: c0374a64ee0840cabe203e97a8ea4e9c (127.27.77.194:35737)
I20260806 00:54:42.655128 7998 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.77.194:46019
W20260806 00:54:42.768321 8440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:42.768586 8440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:42.768647 8440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:42.772747 8440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:42.772807 8440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:42.772890 8440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:54:42.777765 8440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:34699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:36415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:42.778975 8440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:42.780521 8440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:42.790242 8446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:42.790231 8445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:42.790252 8449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:42.790592 8440 server_base.cc:1034] running on GCE node
I20260806 00:54:42.790997 8440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:42.791752 8440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:42.792946 8440 hybrid_clock.cc:648] HybridClock initialized: now 1785977682792916 us; error 47 us; skew 500 ppm
I20260806 00:54:42.795101 8440 webserver.cc:460] Webserver started at http://127.27.77.195:38249/ using document root <none> and password file <none>
I20260806 00:54:42.795773 8440 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:42.795866 8440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:42.796103 8440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:42.798219 8440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "f92773d2061f4e8c81f99e74c9341986"
format_stamp: "Formatted at 2026-08-06 00:54:42 on dist-test-slave-sb2z"
server_key: "4c4f936406694872e7d86ab4779cf8b6"
server_key_iv: "10cc25edac02eab1ae34e4a2df85d7f9"
server_key_version: "encryptionkey@0"
I20260806 00:54:42.798828 8440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "f92773d2061f4e8c81f99e74c9341986"
format_stamp: "Formatted at 2026-08-06 00:54:42 on dist-test-slave-sb2z"
server_key: "4c4f936406694872e7d86ab4779cf8b6"
server_key_iv: "10cc25edac02eab1ae34e4a2df85d7f9"
server_key_version: "encryptionkey@0"
I20260806 00:54:42.802556 8440 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:54:42.805130 8455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:42.806294 8440 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:54:42.806454 8440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "f92773d2061f4e8c81f99e74c9341986"
format_stamp: "Formatted at 2026-08-06 00:54:42 on dist-test-slave-sb2z"
server_key: "4c4f936406694872e7d86ab4779cf8b6"
server_key_iv: "10cc25edac02eab1ae34e4a2df85d7f9"
server_key_version: "encryptionkey@0"
I20260806 00:54:42.806579 8440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:42.825310 8440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:42.826077 8440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:42.826285 8440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:42.827003 8440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:42.828094 8440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:42.828168 8440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:42.828236 8440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:42.828281 8440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:42.837496 8440 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:44927
I20260806 00:54:42.837530 8600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:44927 every 8 connection(s)
I20260806 00:54:42.838438 8440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:54:42.839043 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 8440
I20260806 00:54:42.839164 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:54:42.839443 27959 external_mini_cluster.cc:1428] Setting key 6665b94e2c436258cdf2409e5db6d29c
I20260806 00:54:42.848929 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:42.840366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:38633 (local address 127.27.77.254:36415)
0806 00:54:42.840511 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:42.840516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:42.842309 (+ 1793us) server_negotiation.cc:407] Connection header received
0806 00:54:42.844131 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:42.844135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:42.844194 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:42.844277 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:42.845174 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:42.845702 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:42.846909 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:42.847136 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:42.847249 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:42.848412 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:42.848416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:42.848418 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:42.848437 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:42.848457 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:42.848465 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:42.848540 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:42.848709 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 00:54:42.848777 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 00:54:42.849880 8603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:42.841048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:36415 (local address 127.27.77.195:38633)
0806 00:54:42.842156 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:42.842213 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:42.843891 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:42.844450 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:42.844457 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:42.844539 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:42.845025 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:42.845037 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:42.845834 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:42.845837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:42.846772 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:42.846782 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:42.847294 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:42.847297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:42.847468 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:42.848150 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:54:42.848151 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:42.848192 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:54:42.848256 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:42.848555 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:42.848565 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:42.848571 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 00:54:42.848709 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 00:54:42.849003 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":144,"threads_started":1}
I20260806 00:54:42.851332 8601 heartbeater.cc:347] Connected to a master server at 127.27.77.254:36415
I20260806 00:54:42.851747 8601 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:42.852301 8601 heartbeater.cc:511] Master 127.27.77.254:36415 requested a full tablet report, sending...
I20260806 00:54:42.853494 7998 ts_manager.cc:194] Registered new tserver with Master: f92773d2061f4e8c81f99e74c9341986 (127.27.77.195:44927)
I20260806 00:54:42.854259 7998 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.77.195:38633
I20260806 00:54:42.854671 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 00:54:42.857676 8055 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:42.856695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44374 (local address 127.27.77.254:36415)
0806 00:54:42.856834 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:42.856840 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:42.857029 (+ 189us) server_negotiation.cc:407] Connection header received
0806 00:54:42.857184 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:42.857187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:42.857235 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:42.857316 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:42.857562 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44374: BlockingRecv error: recv got EOF from 127.0.0.1:44374 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:54:42.857901 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 8081
I20260806 00:54:42.880262 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 8267
I20260806 00:54:42.893930 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 8440
I20260806 00:54:42.906881 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 7959
2026-08-06T00:54:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (922 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T00:54:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:42Z Disabled control of system clock
I20260806 00:54:42.945667 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:39647
--webserver_interface=127.27.77.254
--webserver_port=0
--builtin_ntp_servers=127.27.77.212:37239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:39647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 00:54:43.060580 8613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:43.060945 8613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:43.060992 8613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:43.065116 8613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:43.065174 8613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:43.065191 8613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:43.065207 8613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:43.065222 8613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:54:43.069941 8613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:37239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:39647
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:39647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:43.071064 8613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:43.072525 8613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:43.081300 8619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:43.081395 8613 server_base.cc:1034] running on GCE node
W20260806 00:54:43.081311 8618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:43.081326 8622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:43.082183 8613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:43.083333 8613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:43.084529 8613 hybrid_clock.cc:648] HybridClock initialized: now 1785977683084526 us; error 51 us; skew 500 ppm
I20260806 00:54:43.086519 8613 webserver.cc:460] Webserver started at http://127.27.77.254:38873/ using document root <none> and password file <none>
I20260806 00:54:43.087154 8613 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:43.087257 8613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:43.087494 8613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:43.089612 8613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "dfca1ef39df1438196ba51aa9fb73248"
format_stamp: "Formatted at 2026-08-06 00:54:43 on dist-test-slave-sb2z"
server_key: "ce817ffd59137671dda22e90ebdaae6d"
server_key_iv: "1964a0f68a648ceea30b38687eee7b7b"
server_key_version: "encryptionkey@0"
I20260806 00:54:43.090204 8613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "dfca1ef39df1438196ba51aa9fb73248"
format_stamp: "Formatted at 2026-08-06 00:54:43 on dist-test-slave-sb2z"
server_key: "ce817ffd59137671dda22e90ebdaae6d"
server_key_iv: "1964a0f68a648ceea30b38687eee7b7b"
server_key_version: "encryptionkey@0"
I20260806 00:54:43.094062 8613 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 00:54:43.096712 8628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:43.098045 8613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:54:43.098203 8613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "dfca1ef39df1438196ba51aa9fb73248"
format_stamp: "Formatted at 2026-08-06 00:54:43 on dist-test-slave-sb2z"
server_key: "ce817ffd59137671dda22e90ebdaae6d"
server_key_iv: "1964a0f68a648ceea30b38687eee7b7b"
server_key_version: "encryptionkey@0"
I20260806 00:54:43.098332 8613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:43.102996 8613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:43.103614 8613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:43.103804 8613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:43.110596 8613 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:39647
I20260806 00:54:43.110612 8689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:39647 every 8 connection(s)
I20260806 00:54:43.111498 8613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:43.111814 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 8613
I20260806 00:54:43.111907 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:54:43.112277 27959 external_mini_cluster.cc:1428] Setting key e4ab55d773395c5bf78804bac1f08447
I20260806 00:54:43.114630 8690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:43.118384 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:43.113734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38098 (local address 127.27.77.254:39647)
0806 00:54:43.114756 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:43.114775 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:43.114817 (+ 42us) server_negotiation.cc:407] Connection header received
0806 00:54:43.115459 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:43.115475 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:43.115871 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:43.116159 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:43.117048 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:43.117053 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:43.117064 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:43.117084 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:43.117161 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:43.117178 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:43.117323 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:43.117406 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 00:54:43.117677 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":524,"threads_started":1}
I20260806 00:54:43.120699 8690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248: Bootstrap starting.
I20260806 00:54:43.122762 8690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:43.123700 8690 log.cc:824] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:43.125567 8690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248: No bootstrap required, opened a new log
I20260806 00:54:43.129292 8690 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfca1ef39df1438196ba51aa9fb73248" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39647 } }
I20260806 00:54:43.129534 8690 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:43.129596 8690 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfca1ef39df1438196ba51aa9fb73248, State: Initialized, Role: FOLLOWER
I20260806 00:54:43.130019 8690 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfca1ef39df1438196ba51aa9fb73248" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39647 } }
I20260806 00:54:43.130164 8690 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:43.130230 8690 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:43.130328 8690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:43.131191 8690 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfca1ef39df1438196ba51aa9fb73248" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39647 } }
I20260806 00:54:43.131565 8690 leader_election.cc:302] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfca1ef39df1438196ba51aa9fb73248; no voters:
I20260806 00:54:43.131954 8690 leader_election.cc:288] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:54:43.132119 8697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:43.132357 8697 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [term 1 LEADER]: Becoming Leader. State: Replica: dfca1ef39df1438196ba51aa9fb73248, State: Running, Role: LEADER
I20260806 00:54:43.132679 8697 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfca1ef39df1438196ba51aa9fb73248" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39647 } }
I20260806 00:54:43.133209 8690 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:43.134364 8698 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfca1ef39df1438196ba51aa9fb73248. Latest consensus state: current_term: 1 leader_uuid: "dfca1ef39df1438196ba51aa9fb73248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfca1ef39df1438196ba51aa9fb73248" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39647 } } }
I20260806 00:54:43.134492 8698 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:43.134783 8702 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfca1ef39df1438196ba51aa9fb73248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfca1ef39df1438196ba51aa9fb73248" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39647 } } }
I20260806 00:54:43.134889 8702 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:43.134979 8712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:43.138229 8712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:43.143991 8712 catalog_manager.cc:1347] Generated new cluster ID: ea16e5671384490d821aa8e594ac94b1
I20260806 00:54:43.144096 8712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:43.159472 8712 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:54:43.159809 8712 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:43.170727 8712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfca1ef39df1438196ba51aa9fb73248: Generated new TSK 0
I20260806 00:54:43.171478 8712 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:54:43.180326 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39647
--builtin_ntp_servers=127.27.77.212:37239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 00:54:43.295953 8728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:43.296197 8728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:43.296236 8728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:43.300200 8728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:43.300246 8728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:43.300300 8728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:54:43.305148 8728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:37239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:43.306102 8728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:43.307456 8728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:43.316834 8736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:43.316828 8733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:43.316826 8739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:43.317167 8728 server_base.cc:1034] running on GCE node
I20260806 00:54:43.318069 8728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:43.318902 8728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:43.320092 8728 hybrid_clock.cc:648] HybridClock initialized: now 1785977683320067 us; error 58 us; skew 500 ppm
I20260806 00:54:43.322551 8728 webserver.cc:460] Webserver started at http://127.27.77.193:35013/ using document root <none> and password file <none>
I20260806 00:54:43.323307 8728 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:43.323411 8728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:43.323674 8728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:43.326596 8728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "f307a77cf2bd46ddb017b429461daccc"
format_stamp: "Formatted at 2026-08-06 00:54:43 on dist-test-slave-sb2z"
server_key: "34aeab61864952b3e47b8434245f5a91"
server_key_iv: "a09dd9d4f62de4ca45471c8bee41a699"
server_key_version: "encryptionkey@0"
I20260806 00:54:43.327265 8728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "f307a77cf2bd46ddb017b429461daccc"
format_stamp: "Formatted at 2026-08-06 00:54:43 on dist-test-slave-sb2z"
server_key: "34aeab61864952b3e47b8434245f5a91"
server_key_iv: "a09dd9d4f62de4ca45471c8bee41a699"
server_key_version: "encryptionkey@0"
I20260806 00:54:43.331043 8728 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:54:43.333613 8747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:43.334900 8728 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:54:43.335050 8728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "f307a77cf2bd46ddb017b429461daccc"
format_stamp: "Formatted at 2026-08-06 00:54:43 on dist-test-slave-sb2z"
server_key: "34aeab61864952b3e47b8434245f5a91"
server_key_iv: "a09dd9d4f62de4ca45471c8bee41a699"
server_key_version: "encryptionkey@0"
I20260806 00:54:43.335203 8728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:43.339915 8728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:43.340528 8728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:43.340708 8728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:43.341271 8728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:43.342199 8728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:43.342268 8728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:43.342342 8728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:43.342389 8728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:43.352035 8728 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:45711
I20260806 00:54:43.352109 8889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:45711 every 8 connection(s)
I20260806 00:54:43.353057 8728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:54:43.357795 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 8728
I20260806 00:54:43.357961 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:54:43.358377 27959 external_mini_cluster.cc:1428] Setting key 1e84814bac637899ce51ae1e0e7570bb
I20260806 00:54:43.360057 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:43.355349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:39191 (local address 127.27.77.254:39647)
0806 00:54:43.355500 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:43.355504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:43.356776 (+ 1272us) server_negotiation.cc:407] Connection header received
0806 00:54:43.358179 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:43.358183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:43.358236 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:43.358330 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:43.359567 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:43.359571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:43.359574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:43.359588 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:43.359615 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:43.359624 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:43.359718 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:43.359873 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 00:54:43.359920 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:54:43.360544 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39647
--builtin_ntp_servers=127.27.77.212:37239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 00:54:43.361034 8894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:43.355906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:39647 (local address 127.27.77.193:39191)
0806 00:54:43.356621 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:43.356663 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:43.357881 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:43.358503 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:43.358513 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:43.358627 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:43.359277 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:54:43.359278 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:43.359328 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:54:43.359397 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:43.359716 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:43.359727 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:43.359737 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 00:54:43.359917 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 00:54:43.360167 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":125,"threads_started":1}
I20260806 00:54:43.362339 8891 heartbeater.cc:347] Connected to a master server at 127.27.77.254:39647
I20260806 00:54:43.362613 8891 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:43.363103 8891 heartbeater.cc:511] Master 127.27.77.254:39647 requested a full tablet report, sending...
I20260806 00:54:43.364907 8644 ts_manager.cc:194] Registered new tserver with Master: f307a77cf2bd46ddb017b429461daccc (127.27.77.193:45711)
W20260806 00:54:43.477670 8896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:43.477937 8896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:43.477979 8896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:43.482041 8896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:43.482103 8896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:43.482161 8896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:54:43.486824 8896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:37239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:43.487952 8896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:43.489362 8896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:43.499197 8903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:43.499197 8901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:43.499347 8896 server_base.cc:1034] running on GCE node
W20260806 00:54:43.499199 8905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:43.499969 8896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:43.500656 8896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:43.501847 8896 hybrid_clock.cc:648] HybridClock initialized: now 1785977683501824 us; error 31 us; skew 500 ppm
I20260806 00:54:43.503844 8896 webserver.cc:460] Webserver started at http://127.27.77.194:34271/ using document root <none> and password file <none>
I20260806 00:54:43.504431 8896 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:43.504521 8896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:43.504760 8896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:43.506891 8896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "96ff19e1d6ef40a993804e8e816a025a"
format_stamp: "Formatted at 2026-08-06 00:54:43 on dist-test-slave-sb2z"
server_key: "37091088eb819b7d3be6077e99d5a978"
server_key_iv: "32792a023f9d3f93927d25ee94fc21cd"
server_key_version: "encryptionkey@0"
I20260806 00:54:43.507460 8896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "96ff19e1d6ef40a993804e8e816a025a"
format_stamp: "Formatted at 2026-08-06 00:54:43 on dist-test-slave-sb2z"
server_key: "37091088eb819b7d3be6077e99d5a978"
server_key_iv: "32792a023f9d3f93927d25ee94fc21cd"
server_key_version: "encryptionkey@0"
I20260806 00:54:43.511123 8896 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:54:43.513854 8914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:43.515094 8896 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:54:43.515228 8896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "96ff19e1d6ef40a993804e8e816a025a"
format_stamp: "Formatted at 2026-08-06 00:54:43 on dist-test-slave-sb2z"
server_key: "37091088eb819b7d3be6077e99d5a978"
server_key_iv: "32792a023f9d3f93927d25ee94fc21cd"
server_key_version: "encryptionkey@0"
I20260806 00:54:43.515344 8896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:43.520073 8896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:43.520714 8896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:43.520901 8896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:43.521476 8896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:43.522410 8896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:43.522478 8896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:43.522548 8896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:43.522604 8896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:43.531811 8896 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:45239
I20260806 00:54:43.531894 9077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:45239 every 8 connection(s)
I20260806 00:54:43.532867 8896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:54:43.539374 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 8896
I20260806 00:54:43.539564 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:54:43.539968 27959 external_mini_cluster.cc:1428] Setting key 1d233aa2c1abb15711cc2d54b3ff8352
I20260806 00:54:43.540040 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:43.534972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:54719 (local address 127.27.77.254:39647)
0806 00:54:43.535173 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:43.535177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:43.536356 (+ 1179us) server_negotiation.cc:407] Connection header received
0806 00:54:43.537818 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:43.537821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:43.537861 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:43.537928 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:43.539380 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:43.539384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:43.539386 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:43.539400 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:43.539423 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:43.539434 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:43.539623 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:43.539778 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 00:54:43.539821 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:54:43.541105 9080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:43.535387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:39647 (local address 127.27.77.194:54719)
0806 00:54:43.536196 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:43.536243 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:43.537590 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:43.538171 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:43.538181 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:43.538285 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:43.538949 (+ 664us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:54:43.538951 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:43.539063 (+ 112us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:54:43.539175 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:43.539599 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:43.539612 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:43.539620 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 00:54:43.539846 (+ 226us) client_negotiation.cc:240] Negotiation successful
0806 00:54:43.540183 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":182,"threads_started":1}
I20260806 00:54:43.542387 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39647
--builtin_ntp_servers=127.27.77.212:37239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 00:54:43.542606 9078 heartbeater.cc:347] Connected to a master server at 127.27.77.254:39647
I20260806 00:54:43.542872 9078 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:43.543373 9078 heartbeater.cc:511] Master 127.27.77.254:39647 requested a full tablet report, sending...
I20260806 00:54:43.544672 8644 ts_manager.cc:194] Registered new tserver with Master: 96ff19e1d6ef40a993804e8e816a025a (127.27.77.194:45239)
W20260806 00:54:43.665009 9081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:43.665270 9081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:43.665340 9081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:43.669368 9081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:43.669425 9081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:43.669519 9081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:54:43.674327 9081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:37239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:39647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:43.675434 9081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:43.676841 9081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:43.686748 9090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:43.686739 9088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:43.687086 9081 server_base.cc:1034] running on GCE node
W20260806 00:54:43.686739 9086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:43.687709 9081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:43.688370 9081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:43.689566 9081 hybrid_clock.cc:648] HybridClock initialized: now 1785977683689543 us; error 39 us; skew 500 ppm
I20260806 00:54:43.691656 9081 webserver.cc:460] Webserver started at http://127.27.77.195:40993/ using document root <none> and password file <none>
I20260806 00:54:43.692260 9081 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:43.692325 9081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:43.692508 9081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:43.694778 9081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "afce5310f7ab497d9eb6b5d8874c01db"
format_stamp: "Formatted at 2026-08-06 00:54:43 on dist-test-slave-sb2z"
server_key: "a6ef08b4fd97a31719fe36b1f15adbf0"
server_key_iv: "1065bdc3979ebee92520df6fee171118"
server_key_version: "encryptionkey@0"
I20260806 00:54:43.695418 9081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "afce5310f7ab497d9eb6b5d8874c01db"
format_stamp: "Formatted at 2026-08-06 00:54:43 on dist-test-slave-sb2z"
server_key: "a6ef08b4fd97a31719fe36b1f15adbf0"
server_key_iv: "1065bdc3979ebee92520df6fee171118"
server_key_version: "encryptionkey@0"
I20260806 00:54:43.699510 9081 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 00:54:43.702580 9099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:43.704054 9081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260806 00:54:43.704195 9081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "afce5310f7ab497d9eb6b5d8874c01db"
format_stamp: "Formatted at 2026-08-06 00:54:43 on dist-test-slave-sb2z"
server_key: "a6ef08b4fd97a31719fe36b1f15adbf0"
server_key_iv: "1065bdc3979ebee92520df6fee171118"
server_key_version: "encryptionkey@0"
I20260806 00:54:43.704316 9081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:43.709283 9081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:43.709931 9081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:43.710145 9081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:43.710741 9081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:43.711792 9081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:43.711866 9081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:43.711956 9081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:43.712013 9081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:43.722604 9081 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:35983
I20260806 00:54:43.722628 9247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:35983 every 8 connection(s)
I20260806 00:54:43.723644 9081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:54:43.729904 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:43.725585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:60355 (local address 127.27.77.254:39647)
0806 00:54:43.725761 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:43.725766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:43.726823 (+ 1057us) server_negotiation.cc:407] Connection header received
0806 00:54:43.728229 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:43.728232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:43.728270 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:43.728336 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:43.729408 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:43.729411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:43.729413 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:54:43.729426 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:43.729446 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:54:43.729454 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:43.729531 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:43.729661 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 00:54:43.729701 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:54:43.730561 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 9081
I20260806 00:54:43.730649 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:54:43.730840 9250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:43.725970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:39647 (local address 127.27.77.195:60355)
0806 00:54:43.726679 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:43.726721 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:43.728018 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:43.728532 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:43.728539 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:43.728629 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:43.729154 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:54:43.729155 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:43.729195 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:54:43.729265 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:43.729534 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:43.729542 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:43.729547 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 00:54:43.729736 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 00:54:43.729980 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":117,"threads_started":1}
I20260806 00:54:43.730952 27959 external_mini_cluster.cc:1428] Setting key 8cc5229ed7bd893d33d41c9bdb70f1da
I20260806 00:54:43.732023 9248 heartbeater.cc:347] Connected to a master server at 127.27.77.254:39647
I20260806 00:54:43.732276 9248 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:43.732688 9248 heartbeater.cc:511] Master 127.27.77.254:39647 requested a full tablet report, sending...
I20260806 00:54:43.733914 8647 ts_manager.cc:194] Registered new tserver with Master: afce5310f7ab497d9eb6b5d8874c01db (127.27.77.195:35983)
I20260806 00:54:43.744967 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 00:54:43.748040 8695 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:54:43.747097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38100 (local address 127.27.77.254:39647)
0806 00:54:43.747228 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:43.747232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:43.747291 (+ 59us) server_negotiation.cc:407] Connection header received
0806 00:54:43.747394 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:43.747396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:43.747430 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:43.747518 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:43.747868 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38100: BlockingRecv error: recv got EOF from 127.0.0.1:38100 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:54:43.748070 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 8728
I20260806 00:54:43.761746 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 8896
I20260806 00:54:43.774302 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 9081
I20260806 00:54:43.790827 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 8613
2026-08-06T00:54:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (882 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:54:43 dist-test-slave-sb2z krb5kdc[9260](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:54:43 dist-test-slave-sb2z krb5kdc[9260](info): set up 2 sockets
Aug 06 00:54:43 dist-test-slave-sb2z krb5kdc[9260](info): commencing operation
krb5kdc: starting...
W20260806 00:54:44.995783 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:54:45 dist-test-slave-sb2z krb5kdc[9260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:54:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:54:45.127236 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:33715
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:33715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:45.238395 9276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:45.238659 9276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:45.238699 9276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:45.242570 9276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:54:45.242619 9276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:45.242637 9276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:45.242654 9276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:54:45.242671 9276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:54:45.247828 9276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:33715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:33715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:45.248878 9276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:45.249912 9276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:45.258570 9284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:45.258528 9288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:45.258535 9283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:45.258582 9276 server_base.cc:1034] running on GCE node
I20260806 00:54:45.259418 9276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:45.260571 9276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:45.261754 9276 hybrid_clock.cc:648] HybridClock initialized: now 1785977685261733 us; error 34 us; skew 500 ppm
Aug 06 00:54:45 dist-test-slave-sb2z krb5kdc[9260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977685, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:45.272505 9276 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:54:45.273485 9276 webserver.cc:460] Webserver started at http://127.27.77.254:43645/ using document root <none> and password file <none>
I20260806 00:54:45.274135 9276 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:45.274238 9276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:45.274493 9276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:45.276782 9276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "1dfcda7f8ed945f28d19b7d2b60676d3"
format_stamp: "Formatted at 2026-08-06 00:54:45 on dist-test-slave-sb2z"
server_key: "109624b5a34a158175a60d98bf6f162b"
server_key_iv: "4697cc049da7ca95290a849f086df85e"
server_key_version: "encryptionkey@0"
I20260806 00:54:45.277410 9276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "1dfcda7f8ed945f28d19b7d2b60676d3"
format_stamp: "Formatted at 2026-08-06 00:54:45 on dist-test-slave-sb2z"
server_key: "109624b5a34a158175a60d98bf6f162b"
server_key_iv: "4697cc049da7ca95290a849f086df85e"
server_key_version: "encryptionkey@0"
I20260806 00:54:45.281596 9276 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:54:45.284286 9296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:45.285579 9276 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 00:54:45.285739 9276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "1dfcda7f8ed945f28d19b7d2b60676d3"
format_stamp: "Formatted at 2026-08-06 00:54:45 on dist-test-slave-sb2z"
server_key: "109624b5a34a158175a60d98bf6f162b"
server_key_iv: "4697cc049da7ca95290a849f086df85e"
server_key_version: "encryptionkey@0"
I20260806 00:54:45.285864 9276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:45.328151 9276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:45.332062 9276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:45.332316 9276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:45.340588 9276 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:33715
I20260806 00:54:45.340658 9368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:33715 every 8 connection(s)
I20260806 00:54:45.341737 9276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:54:45.344478 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 9276
I20260806 00:54:45.344627 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:54:45.344955 27959 external_mini_cluster.cc:1428] Setting key 3abc0e9f89603fab5f8c27b295453c01
I20260806 00:54:45.345149 9369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:45.352159 9369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3: Bootstrap starting.
Aug 06 00:54:45 dist-test-slave-sb2z krb5kdc[9260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:45.355273 9369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:45.356299 9369 log.cc:824] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:45.358379 9369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3: No bootstrap required, opened a new log
I20260806 00:54:45.360452 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:45.346560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49104 (local address 127.27.77.254:33715)
0806 00:54:45.347022 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:45.347038 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:45.347074 (+ 36us) server_negotiation.cc:407] Connection header received
0806 00:54:45.348071 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:45.348091 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:45.348476 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:45.348848 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:45.349702 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:45.350738 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:45.351727 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:45.352276 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:45.352429 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:45.355478 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:45.355484 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:45.355498 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:45.355580 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:45.357473 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:45.357975 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:45.357980 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:45.357985 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:45.358099 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:45.358397 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:45.358401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:45.358403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:45.358880 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:45.359100 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:45.359384 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 00:54:45.359725 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":139,"threads_started":1}
I20260806 00:54:45.362181 9369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dfcda7f8ed945f28d19b7d2b60676d3" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33715 } }
I20260806 00:54:45.362435 9369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:45.362532 9369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dfcda7f8ed945f28d19b7d2b60676d3, State: Initialized, Role: FOLLOWER
I20260806 00:54:45.362963 9369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dfcda7f8ed945f28d19b7d2b60676d3" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33715 } }
I20260806 00:54:45.363094 9369 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:54:45.363142 9369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:54:45.363212 9369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:45.364187 9369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dfcda7f8ed945f28d19b7d2b60676d3" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33715 } }
I20260806 00:54:45.364496 9369 leader_election.cc:302] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dfcda7f8ed945f28d19b7d2b60676d3; no voters:
I20260806 00:54:45.364760 9369 leader_election.cc:288] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:54:45.364912 9378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:45.365243 9378 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [term 1 LEADER]: Becoming Leader. State: Replica: 1dfcda7f8ed945f28d19b7d2b60676d3, State: Running, Role: LEADER
I20260806 00:54:45.365566 9378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dfcda7f8ed945f28d19b7d2b60676d3" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33715 } }
I20260806 00:54:45.366112 9369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:54:45.367327 9380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dfcda7f8ed945f28d19b7d2b60676d3. Latest consensus state: current_term: 1 leader_uuid: "1dfcda7f8ed945f28d19b7d2b60676d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dfcda7f8ed945f28d19b7d2b60676d3" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33715 } } }
I20260806 00:54:45.367477 9380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [sys.catalog]: This master's current role is: LEADER
I20260806 00:54:45.367864 9379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dfcda7f8ed945f28d19b7d2b60676d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dfcda7f8ed945f28d19b7d2b60676d3" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33715 } } }
I20260806 00:54:45.367985 9379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3 [sys.catalog]: This master's current role is: LEADER
W20260806 00:54:45.371208 9395 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:54:45.371308 9395 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:54:45.371464 9388 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:54:45.372300 9388 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:54:45.378440 9388 catalog_manager.cc:1347] Generated new cluster ID: 08e231bd62c04addb2b06670dd502b3b
I20260806 00:54:45.378516 9388 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:54:45.389166 9388 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:54:45.390475 9388 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:54:45.411459 9388 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3: Generated new TSK 0
I20260806 00:54:45.412343 9388 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:54:45.459036 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33715
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:45.597755 9401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:45.598011 9401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:45.598053 9401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:45.602164 9401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:45.602226 9401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:45.602310 9401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:54:45.607738 9401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:45.608832 9401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:45.609894 9401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:45.619722 9406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:45.619745 9407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:45.619771 9401 server_base.cc:1034] running on GCE node
W20260806 00:54:45.619722 9410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:45.620452 9401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:45.621147 9401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:45.622311 9401 hybrid_clock.cc:648] HybridClock initialized: now 1785977685622275 us; error 53 us; skew 500 ppm
Aug 06 00:54:45 dist-test-slave-sb2z krb5kdc[9260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977685, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:45.625355 9401 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:54:45.626320 9401 webserver.cc:460] Webserver started at http://127.27.77.193:46521/ using document root <none> and password file <none>
I20260806 00:54:45.626895 9401 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:45.626959 9401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:45.627187 9401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:45.629626 9401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "88a461eeb55b4f05ac2719ca97d06491"
format_stamp: "Formatted at 2026-08-06 00:54:45 on dist-test-slave-sb2z"
server_key: "86d085bf5f7b7b9973dd599a51b621dd"
server_key_iv: "0c980363f32d8a6f91f8077df719906d"
server_key_version: "encryptionkey@0"
I20260806 00:54:45.630291 9401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "88a461eeb55b4f05ac2719ca97d06491"
format_stamp: "Formatted at 2026-08-06 00:54:45 on dist-test-slave-sb2z"
server_key: "86d085bf5f7b7b9973dd599a51b621dd"
server_key_iv: "0c980363f32d8a6f91f8077df719906d"
server_key_version: "encryptionkey@0"
I20260806 00:54:45.634447 9401 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 00:54:45.637142 9418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:45.638386 9401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:54:45.638541 9401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "88a461eeb55b4f05ac2719ca97d06491"
format_stamp: "Formatted at 2026-08-06 00:54:45 on dist-test-slave-sb2z"
server_key: "86d085bf5f7b7b9973dd599a51b621dd"
server_key_iv: "0c980363f32d8a6f91f8077df719906d"
server_key_version: "encryptionkey@0"
I20260806 00:54:45.638670 9401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:45.653755 9401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:45.657150 9401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:45.657351 9401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:45.657951 9401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:45.658815 9401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:45.658896 9401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:45.659024 9401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:45.659152 9401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:45.669315 9401 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:46425
I20260806 00:54:45.669333 9564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:46425 every 8 connection(s)
I20260806 00:54:45.670259 9401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:54:45.677098 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 9401
I20260806 00:54:45.677248 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:54:45.677552 27959 external_mini_cluster.cc:1428] Setting key acfaaf95755151b359f773b07b9c0bf7
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:54:45.706626 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33715
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:45.823454 9573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:45.823770 9573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:45.823817 9573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:45.827898 9573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:45.827952 9573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:45.828011 9573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:54:45.833273 9573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:45.834316 9573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:45.835379 9573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:45.845371 9582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:45.845440 9584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:45.845494 9581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:45.845621 9573 server_base.cc:1034] running on GCE node
I20260806 00:54:45.846063 9573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:45.846690 9573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:45.847880 9573 hybrid_clock.cc:648] HybridClock initialized: now 1785977685847858 us; error 34 us; skew 500 ppm
W20260806 00:54:46.537556 9570 sasl_common.cc:323] Time spent creating new SASL kudu client: real 0.859s user 0.001s sys 0.000s
W20260806 00:54:46.537536 9573 net_util.cc:504] Time spent look up canonical hostname for localhost 'dist-test-slave-sb2z': real 0.689s user 0.001s sys 0.000s
Aug 06 00:54:46 dist-test-slave-sb2z krb5kdc[9260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977685, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
Aug 06 00:54:46 dist-test-slave-sb2z krb5kdc[9260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977686, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:46.541630 9573 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:54:46.544032 9573 webserver.cc:460] Webserver started at http://127.27.77.194:39501/ using document root <none> and password file <none>
I20260806 00:54:46.545075 9573 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:46.545228 9573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:46.545707 9573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:46.549077 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:45.672221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:55839 (local address 127.27.77.254:33715)
0806 00:54:45.672365 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:45.672369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:45.673287 (+ 918us) server_negotiation.cc:407] Connection header received
0806 00:54:45.674701 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:45.674704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:45.674754 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:45.674836 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:45.676100 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:45.676688 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:45.677937 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:45.678293 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:45.678410 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:46.541639 (+863229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:46.541644 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:46.541646 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:46.541696 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:46.544938 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:46.546120 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:46.546124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:46.546126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:46.546280 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:46.546875 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:46.546883 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:46.546885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:46.547226 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:46.547433 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:46.548395 (+ 962us) server_negotiation.cc:299] Negotiation successful
0806 00:54:46.548712 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:54:46.549456 9573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "b49f68da0e5d4d00b56c3c0d5e62932b"
format_stamp: "Formatted at 2026-08-06 00:54:46 on dist-test-slave-sb2z"
server_key: "fc8e34a60061dd707ed3b6c61f86a336"
server_key_iv: "0ba7ca749b6b1e05cce5b2b9cb291a86"
server_key_version: "encryptionkey@0"
I20260806 00:54:46.550491 9570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:45.672672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:33715 (local address 127.27.77.193:55839)
0806 00:54:45.673134 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:45.673177 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:45.674498 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:45.674993 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:45.675000 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:45.675391 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:45.675954 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:45.675968 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:45.676804 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:45.676807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:45.677809 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:45.677819 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:45.678465 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:45.678469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:45.678676 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:46.537788 (+859112us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:46.537850 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:46.541233 (+ 3383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:46.545200 (+ 3967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:46.545238 (+ 38us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:46.545274 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:46.545915 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:46.546439 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:46.546446 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:46.546448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:46.546686 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:46.547436 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:46.547444 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:46.547977 (+ 533us) client_negotiation.cc:769] Sending connection context
0806 00:54:46.548308 (+ 331us) client_negotiation.cc:240] Negotiation successful
0806 00:54:46.549103 (+ 795us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":119,"threads_started":1}
I20260806 00:54:46.550504 9573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "b49f68da0e5d4d00b56c3c0d5e62932b"
format_stamp: "Formatted at 2026-08-06 00:54:46 on dist-test-slave-sb2z"
server_key: "fc8e34a60061dd707ed3b6c61f86a336"
server_key_iv: "0ba7ca749b6b1e05cce5b2b9cb291a86"
server_key_version: "encryptionkey@0"
I20260806 00:54:46.552969 9566 heartbeater.cc:347] Connected to a master server at 127.27.77.254:33715
I20260806 00:54:46.553393 9566 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:46.554244 9566 heartbeater.cc:511] Master 127.27.77.254:33715 requested a full tablet report, sending...
I20260806 00:54:46.556742 9317 ts_manager.cc:194] Registered new tserver with Master: 88a461eeb55b4f05ac2719ca97d06491 (127.27.77.193:46425)
I20260806 00:54:46.558480 9317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:55839
I20260806 00:54:46.558569 9573 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 00:54:46.562994 9593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:46.564985 9573 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:54:46.565156 9573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "b49f68da0e5d4d00b56c3c0d5e62932b"
format_stamp: "Formatted at 2026-08-06 00:54:46 on dist-test-slave-sb2z"
server_key: "fc8e34a60061dd707ed3b6c61f86a336"
server_key_iv: "0ba7ca749b6b1e05cce5b2b9cb291a86"
server_key_version: "encryptionkey@0"
I20260806 00:54:46.565321 9573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:46.583655 9573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:46.589658 9573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:46.590023 9573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:46.591039 9573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:46.592518 9573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:46.592638 9573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:46.592777 9573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:46.592852 9573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:46.609448 9573 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:37957
I20260806 00:54:46.609510 9746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:37957 every 8 connection(s)
I20260806 00:54:46.611012 9573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:54:46.619318 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 9573
I20260806 00:54:46.619620 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:54:46.620055 27959 external_mini_cluster.cc:1428] Setting key d6a41e8c2a4bf75a54f99cec35ac891c
Aug 06 00:54:46 dist-test-slave-sb2z krb5kdc[9260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977686, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:46.638633 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:46.614348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:44551 (local address 127.27.77.254:33715)
0806 00:54:46.614601 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:46.614614 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:46.616000 (+ 1386us) server_negotiation.cc:407] Connection header received
0806 00:54:46.618665 (+ 2665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:46.618669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:46.618779 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:46.619013 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:46.621921 (+ 2908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:46.622884 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:46.625224 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:46.625722 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:46.625934 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:46.630361 (+ 4427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:46.630379 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:46.630388 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:46.630461 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:46.634136 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:46.635144 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:46.635149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:46.635153 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:46.635343 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:46.636054 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:46.636058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:46.636060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:46.636600 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:46.636819 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:46.637848 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0806 00:54:46.638201 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260806 00:54:46.639739 9749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:46.614861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:33715 (local address 127.27.77.194:44551)
0806 00:54:46.615742 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:46.615813 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:46.618227 (+ 2414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:46.619398 (+ 1171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:46.619416 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:46.620377 (+ 961us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:46.621602 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:46.621635 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:46.623115 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:46.623132 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:46.624930 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:46.624943 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:46.626370 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:46.626378 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:46.626621 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:46.627741 (+ 1120us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:46.627771 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:46.630025 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:46.634391 (+ 4366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:46.634399 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:46.634422 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:46.634971 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:46.635567 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:46.635584 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:46.635589 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:46.635838 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:46.636787 (+ 949us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:46.636801 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:46.637267 (+ 466us) client_negotiation.cc:769] Sending connection context
0806 00:54:46.637646 (+ 379us) client_negotiation.cc:240] Negotiation successful
0806 00:54:46.638150 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":171,"threads_started":1}
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
I20260806 00:54:46.641691 9747 heartbeater.cc:347] Connected to a master server at 127.27.77.254:33715
I20260806 00:54:46.642153 9747 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:46.643430 9747 heartbeater.cc:511] Master 127.27.77.254:33715 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
I20260806 00:54:46.645498 9317 ts_manager.cc:194] Registered new tserver with Master: b49f68da0e5d4d00b56c3c0d5e62932b (127.27.77.194:37957)
I20260806 00:54:46.646411 9317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:44551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:54:46.668808 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33715
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:54:46.908496 9752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:54:46.908983 9752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:54:46.909065 9752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:54:46.915163 9752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:54:46.915275 9752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:54:46.915365 9752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:54:46.923718 9752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:54:46.925557 9752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:54:46.927284 9752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:54:46.944883 9760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:46.945045 9763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:54:46.945000 9761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:54:46.945349 9752 server_base.cc:1034] running on GCE node
I20260806 00:54:46.946231 9752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:54:46.947275 9752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:54:46.948503 9752 hybrid_clock.cc:648] HybridClock initialized: now 1785977686948429 us; error 74 us; skew 500 ppm
Aug 06 00:54:46 dist-test-slave-sb2z krb5kdc[9260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977686, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:54:46.953406 9752 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:54:46.954963 9752 webserver.cc:460] Webserver started at http://127.27.77.195:38033/ using document root <none> and password file <none>
I20260806 00:54:46.955973 9752 fs_manager.cc:359] Metadata directory not provided
I20260806 00:54:46.956090 9752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:54:46.956444 9752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:54:46.959779 9752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "b424b51e9d974b2aa86ea602c8d40edb"
format_stamp: "Formatted at 2026-08-06 00:54:46 on dist-test-slave-sb2z"
server_key: "3f12e1f90181ffd30e71049c939ea607"
server_key_iv: "229ee6563ceed70e969f07694a97d568"
server_key_version: "encryptionkey@0"
I20260806 00:54:46.961113 9752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "b424b51e9d974b2aa86ea602c8d40edb"
format_stamp: "Formatted at 2026-08-06 00:54:46 on dist-test-slave-sb2z"
server_key: "3f12e1f90181ffd30e71049c939ea607"
server_key_iv: "229ee6563ceed70e969f07694a97d568"
server_key_version: "encryptionkey@0"
I20260806 00:54:46.968447 9752 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 00:54:46.973402 9774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:46.975854 9752 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 00:54:46.976058 9752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "b424b51e9d974b2aa86ea602c8d40edb"
format_stamp: "Formatted at 2026-08-06 00:54:46 on dist-test-slave-sb2z"
server_key: "3f12e1f90181ffd30e71049c939ea607"
server_key_iv: "229ee6563ceed70e969f07694a97d568"
server_key_version: "encryptionkey@0"
I20260806 00:54:46.976269 9752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:54:47.032994 9752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:54:47.037652 9752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:54:47.037973 9752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:54:47.038996 9752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:54:47.040283 9752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:54:47.040400 9752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:47.040560 9752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:54:47.040654 9752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:54:47.056448 9752 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:44263
I20260806 00:54:47.056497 9919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:44263 every 8 connection(s)
I20260806 00:54:47.057811 9752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:54:47.064847 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 9752
I20260806 00:54:47.065030 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:54:47.065526 27959 external_mini_cluster.cc:1428] Setting key 1538cbd32babd5f9245b2eb6b9b48c2d
Aug 06 00:54:47 dist-test-slave-sb2z krb5kdc[9260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977686, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:54:47.078877 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:47.060647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:45365 (local address 127.27.77.254:33715)
0806 00:54:47.060890 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:47.060900 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:47.062537 (+ 1637us) server_negotiation.cc:407] Connection header received
0806 00:54:47.064635 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:47.064642 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:47.064749 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:47.064881 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:47.067111 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.068107 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.069968 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.070302 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.070439 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:47.074102 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:47.074109 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:47.074112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:47.074147 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:47.076315 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:47.077016 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:47.077019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:47.077020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:47.077131 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:47.077556 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:47.077558 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:47.077559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:47.077803 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:47.077911 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:47.078612 (+ 701us) server_negotiation.cc:299] Negotiation successful
0806 00:54:47.078734 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 00:54:47.080063 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:47.061298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:33715 (local address 127.27.77.195:45365)
0806 00:54:47.062339 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:47.062400 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:47.064278 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:47.065136 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:47.065150 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:47.065927 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:54:47.066809 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:47.066854 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.068272 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.068276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:47.069774 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:47.069792 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.070501 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.070507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:47.070733 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:47.071470 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:54:47.071499 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:54:47.073900 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:54:47.076442 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:47.076448 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:47.076466 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:47.076866 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:47.077252 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:54:47.077255 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:54:47.077256 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:54:47.077457 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:54:47.077936 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:54:47.077947 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:54:47.078287 (+ 340us) client_negotiation.cc:769] Sending connection context
0806 00:54:47.078523 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 00:54:47.078833 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":222,"threads_started":1}
I20260806 00:54:47.081674 9922 heartbeater.cc:347] Connected to a master server at 127.27.77.254:33715
I20260806 00:54:47.081928 9922 heartbeater.cc:464] Registering TS with master...
I20260806 00:54:47.082625 9922 heartbeater.cc:511] Master 127.27.77.254:33715 requested a full tablet report, sending...
I20260806 00:54:47.084035 9316 ts_manager.cc:194] Registered new tserver with Master: b424b51e9d974b2aa86ea602c8d40edb (127.27.77.195:44263)
I20260806 00:54:47.084880 9316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:45365
I20260806 00:54:47.093360 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:54:47.106063 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:47.095901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49106 (local address 127.27.77.254:33715)
0806 00:54:47.096092 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:47.096101 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:47.096438 (+ 337us) server_negotiation.cc:407] Connection header received
0806 00:54:47.096661 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:47.096664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:47.096735 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:47.096842 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:54:47.097833 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.098479 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.099627 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.099922 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.100091 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:47.101874 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:54:47.101881 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:54:47.101884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:54:47.101924 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:54:47.103638 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:47.104229 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:47.104236 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:47.104243 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:47.104360 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:54:47.104759 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:54:47.104762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:54:47.104764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:54:47.105055 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:54:47.105255 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:47.105601 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 00:54:47.105825 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:54:47.110195 9316 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:54:47.114645 9316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:54:47.134274 9946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:47.126070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:46425 (local address 127.0.0.1:55836)
0806 00:54:47.127604 (+ 1534us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:47.127624 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:47.127829 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:47.128540 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:47.128543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:47.128566 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:47.128795 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:47.128819 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.130251 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.130254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:47.131880 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:47.131892 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.133458 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.133462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:47.133625 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:47.133670 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 00:54:47.133825 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 00:54:47.134007 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1401,"mutex_wait_us":234,"thread_start_us":152,"threads_started":1}
I20260806 00:54:47.135121 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:47.125509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:44263 (local address 127.0.0.1:36960)
0806 00:54:47.126894 (+ 1385us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:47.126921 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:47.127166 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:47.127947 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:47.127951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:47.127976 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:47.128146 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:47.128153 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.130229 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.130233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:47.132747 (+ 2514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:47.132763 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.134553 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.134557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:47.134698 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:47.134737 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 00:54:47.134839 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 00:54:47.134929 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1297,"mutex_wait_us":451,"thread_start_us":130,"threads_started":1}
I20260806 00:54:47.135191 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:47.125390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:37957 (local address 127.0.0.1:38300)
0806 00:54:47.125965 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:47.126015 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:47.126194 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:47.127653 (+ 1459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:47.127660 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:47.127689 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:47.127903 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:47.127918 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.130229 (+ 2311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.130233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:47.132747 (+ 2514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:47.132763 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.134601 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.134615 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:47.134778 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:47.134810 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 00:54:47.134908 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 00:54:47.134995 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":465,"thread_start_us":1227,"threads_started":1}
I20260806 00:54:47.135169 9949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:47.126746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55836 (local address 127.27.77.193:46425)
0806 00:54:47.127359 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:47.127371 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:47.127788 (+ 417us) server_negotiation.cc:407] Connection header received
0806 00:54:47.128095 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:47.128104 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:47.128393 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:47.128568 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:47.128985 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.130078 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.132046 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.132816 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.133045 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:47.133336 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:47.134783 (+ 1447us) server_negotiation.cc:299] Negotiation successful
0806 00:54:47.134953 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":143,"threads_started":1}
I20260806 00:54:47.135811 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:47.126085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36960 (local address 127.27.77.195:44263)
0806 00:54:47.126691 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:47.126705 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:47.127025 (+ 320us) server_negotiation.cc:407] Connection header received
0806 00:54:47.127378 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:47.127388 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:47.127685 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:47.127863 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:47.128361 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.130015 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.132986 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.134063 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.134206 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:47.134434 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:47.134883 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 00:54:47.135005 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":188,"threads_started":1}
I20260806 00:54:47.135861 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:47.125575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38300 (local address 127.27.77.194:37957)
0806 00:54:47.126791 (+ 1216us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:47.126816 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:47.126842 (+ 26us) server_negotiation.cc:407] Connection header received
0806 00:54:47.126984 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:47.127004 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:47.127354 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:47.128135 (+ 781us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:47.128400 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.130025 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.132963 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.134088 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.134251 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:47.134508 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:47.135451 (+ 943us) server_negotiation.cc:299] Negotiation successful
0806 00:54:47.135640 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1054,"thread_start_us":148,"threads_started":1}
I20260806 00:54:47.139662 9479 tablet_service.cc:1467] Processing CreateTablet for tablet 8a31b352a4654822b77bb587c5587572 (DEFAULT_TABLE table=test-table [id=0e935715e4b94f97b7de61b80625fcc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:47.139642 9841 tablet_service.cc:1467] Processing CreateTablet for tablet 8a31b352a4654822b77bb587c5587572 (DEFAULT_TABLE table=test-table [id=0e935715e4b94f97b7de61b80625fcc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:47.139642 9664 tablet_service.cc:1467] Processing CreateTablet for tablet 8a31b352a4654822b77bb587c5587572 (DEFAULT_TABLE table=test-table [id=0e935715e4b94f97b7de61b80625fcc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:54:47.141199 9664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a31b352a4654822b77bb587c5587572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:47.141251 9841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a31b352a4654822b77bb587c5587572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:47.141251 9479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a31b352a4654822b77bb587c5587572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:54:47.150084 9951 tablet_bootstrap.cc:492] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b: Bootstrap starting.
I20260806 00:54:47.152357 9954 tablet_bootstrap.cc:492] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb: Bootstrap starting.
I20260806 00:54:47.152357 9953 tablet_bootstrap.cc:492] T 8a31b352a4654822b77bb587c5587572 P 88a461eeb55b4f05ac2719ca97d06491: Bootstrap starting.
I20260806 00:54:47.154353 9951 tablet_bootstrap.cc:654] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:47.155696 9951 log.cc:824] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:47.155722 9953 tablet_bootstrap.cc:654] T 8a31b352a4654822b77bb587c5587572 P 88a461eeb55b4f05ac2719ca97d06491: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:47.156471 9954 tablet_bootstrap.cc:654] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb: Neither blocks nor log segments found. Creating new log.
I20260806 00:54:47.157179 9953 log.cc:824] T 8a31b352a4654822b77bb587c5587572 P 88a461eeb55b4f05ac2719ca97d06491: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:47.157760 9954 log.cc:824] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb: Log is configured to *not* fsync() on all Append() calls
I20260806 00:54:47.159149 9951 tablet_bootstrap.cc:492] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b: No bootstrap required, opened a new log
I20260806 00:54:47.159423 9951 ts_tablet_manager.cc:1397] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260806 00:54:47.159870 9953 tablet_bootstrap.cc:492] T 8a31b352a4654822b77bb587c5587572 P 88a461eeb55b4f05ac2719ca97d06491: No bootstrap required, opened a new log
I20260806 00:54:47.160122 9953 ts_tablet_manager.cc:1397] T 8a31b352a4654822b77bb587c5587572 P 88a461eeb55b4f05ac2719ca97d06491: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260806 00:54:47.162051 9954 tablet_bootstrap.cc:492] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb: No bootstrap required, opened a new log
I20260806 00:54:47.162460 9954 ts_tablet_manager.cc:1397] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260806 00:54:47.164495 9953 raft_consensus.cc:348] T 8a31b352a4654822b77bb587c5587572 P 88a461eeb55b4f05ac2719ca97d06491 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49f68da0e5d4d00b56c3c0d5e62932b" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37957 } } peers { permanent_uuid: "b424b51e9d974b2aa86ea602c8d40edb" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44263 } } peers { permanent_uuid: "88a461eeb55b4f05ac2719ca97d06491" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46425 } }
I20260806 00:54:47.164865 9953 raft_consensus.cc:374] T 8a31b352a4654822b77bb587c5587572 P 88a461eeb55b4f05ac2719ca97d06491 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:47.164922 9953 raft_consensus.cc:729] T 8a31b352a4654822b77bb587c5587572 P 88a461eeb55b4f05ac2719ca97d06491 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a461eeb55b4f05ac2719ca97d06491, State: Initialized, Role: FOLLOWER
I20260806 00:54:47.165478 9953 consensus_queue.cc:260] T 8a31b352a4654822b77bb587c5587572 P 88a461eeb55b4f05ac2719ca97d06491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49f68da0e5d4d00b56c3c0d5e62932b" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37957 } } peers { permanent_uuid: "b424b51e9d974b2aa86ea602c8d40edb" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44263 } } peers { permanent_uuid: "88a461eeb55b4f05ac2719ca97d06491" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46425 } }
I20260806 00:54:47.166293 9566 heartbeater.cc:503] Master 127.27.77.254:33715 was elected leader, sending a full tablet report...
I20260806 00:54:47.166949 9953 ts_tablet_manager.cc:1428] T 8a31b352a4654822b77bb587c5587572 P 88a461eeb55b4f05ac2719ca97d06491: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 00:54:47.166945 9951 raft_consensus.cc:348] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49f68da0e5d4d00b56c3c0d5e62932b" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37957 } } peers { permanent_uuid: "b424b51e9d974b2aa86ea602c8d40edb" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44263 } } peers { permanent_uuid: "88a461eeb55b4f05ac2719ca97d06491" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46425 } }
I20260806 00:54:47.167421 9951 raft_consensus.cc:374] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:47.167587 9951 raft_consensus.cc:729] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b49f68da0e5d4d00b56c3c0d5e62932b, State: Initialized, Role: FOLLOWER
I20260806 00:54:47.168780 9951 consensus_queue.cc:260] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49f68da0e5d4d00b56c3c0d5e62932b" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37957 } } peers { permanent_uuid: "b424b51e9d974b2aa86ea602c8d40edb" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44263 } } peers { permanent_uuid: "88a461eeb55b4f05ac2719ca97d06491" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46425 } }
I20260806 00:54:47.169922 9747 heartbeater.cc:503] Master 127.27.77.254:33715 was elected leader, sending a full tablet report...
I20260806 00:54:47.170651 9951 ts_tablet_manager.cc:1428] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b: Time spent starting tablet: real 0.011s user 0.005s sys 0.004s
I20260806 00:54:47.170449 9954 raft_consensus.cc:348] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49f68da0e5d4d00b56c3c0d5e62932b" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37957 } } peers { permanent_uuid: "b424b51e9d974b2aa86ea602c8d40edb" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44263 } } peers { permanent_uuid: "88a461eeb55b4f05ac2719ca97d06491" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46425 } }
I20260806 00:54:47.170928 9954 raft_consensus.cc:374] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:54:47.171032 9954 raft_consensus.cc:729] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b424b51e9d974b2aa86ea602c8d40edb, State: Initialized, Role: FOLLOWER
I20260806 00:54:47.171854 9954 consensus_queue.cc:260] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49f68da0e5d4d00b56c3c0d5e62932b" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37957 } } peers { permanent_uuid: "b424b51e9d974b2aa86ea602c8d40edb" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44263 } } peers { permanent_uuid: "88a461eeb55b4f05ac2719ca97d06491" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46425 } }
I20260806 00:54:47.172835 9922 heartbeater.cc:503] Master 127.27.77.254:33715 was elected leader, sending a full tablet report...
W20260806 00:54:47.172902 9569 tablet.cc:2369] T 8a31b352a4654822b77bb587c5587572 P 88a461eeb55b4f05ac2719ca97d06491: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:54:47.173409 9954 ts_tablet_manager.cc:1428] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
W20260806 00:54:47.309387 9924 tablet.cc:2369] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:54:47.363133 9748 tablet.cc:2369] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:54:47.367731 9962 raft_consensus.cc:482] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:54:47.367972 9962 raft_consensus.cc:504] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49f68da0e5d4d00b56c3c0d5e62932b" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37957 } } peers { permanent_uuid: "b424b51e9d974b2aa86ea602c8d40edb" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44263 } } peers { permanent_uuid: "88a461eeb55b4f05ac2719ca97d06491" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46425 } }
I20260806 00:54:47.369750 9962 leader_election.cc:288] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b49f68da0e5d4d00b56c3c0d5e62932b (127.27.77.194:37957), 88a461eeb55b4f05ac2719ca97d06491 (127.27.77.193:46425)
I20260806 00:54:47.376991 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:47.370426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:52563 (local address 127.27.77.194:37957)
0806 00:54:47.370686 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:47.370694 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:47.370752 (+ 58us) server_negotiation.cc:407] Connection header received
0806 00:54:47.370831 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:47.370833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:47.370891 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:47.370974 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:47.371498 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.372705 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.374938 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.375980 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.376181 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:47.376265 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:47.376681 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 00:54:47.376784 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260806 00:54:47.376991 9949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:47.370479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:41093 (local address 127.27.77.193:46425)
0806 00:54:47.370670 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:47.370676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:47.371074 (+ 398us) server_negotiation.cc:407] Connection header received
0806 00:54:47.371271 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:47.371273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:47.371333 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:47.371479 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:54:47.371858 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.372971 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.374950 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.376027 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.376183 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:47.376265 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:47.376681 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 00:54:47.376791 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 00:54:47.377040 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:47.370084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:37957 (local address 127.27.77.195:52563)
0806 00:54:47.370379 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:47.370400 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:47.370627 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:47.371036 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:47.371039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:47.371101 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:47.371293 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:47.371302 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.372875 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.372878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:47.374737 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:47.374754 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.376196 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.376200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:47.376428 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:47.376513 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 00:54:47.376617 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 00:54:47.376816 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":201,"spinlock_wait_cycles":43648}
I20260806 00:54:47.377053 9963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:47.370135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:46425 (local address 127.27.77.195:41093)
0806 00:54:47.370906 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:47.370918 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:54:47.371112 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:54:47.371566 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:54:47.371569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:54:47.371601 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:54:47.371739 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:47.371747 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.373088 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.373091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:47.374745 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:54:47.374755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.376196 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.376200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:54:47.376424 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:47.376517 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 00:54:47.376617 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 00:54:47.376816 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":126,"spinlock_wait_cycles":5760,"thread_start_us":210,"threads_started":1}
I20260806 00:54:47.378260 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a31b352a4654822b77bb587c5587572" candidate_uuid: "b424b51e9d974b2aa86ea602c8d40edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b49f68da0e5d4d00b56c3c0d5e62932b" is_pre_election: true
I20260806 00:54:47.378266 9504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a31b352a4654822b77bb587c5587572" candidate_uuid: "b424b51e9d974b2aa86ea602c8d40edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a461eeb55b4f05ac2719ca97d06491" is_pre_election: true
I20260806 00:54:47.378671 9688 raft_consensus.cc:2454] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b424b51e9d974b2aa86ea602c8d40edb in term 0.
I20260806 00:54:47.378719 9504 raft_consensus.cc:2454] T 8a31b352a4654822b77bb587c5587572 P 88a461eeb55b4f05ac2719ca97d06491 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b424b51e9d974b2aa86ea602c8d40edb in term 0.
I20260806 00:54:47.379427 9777 leader_election.cc:302] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b424b51e9d974b2aa86ea602c8d40edb, b49f68da0e5d4d00b56c3c0d5e62932b; no voters:
I20260806 00:54:47.379894 9962 raft_consensus.cc:2781] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:54:47.380050 9962 raft_consensus.cc:482] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:54:47.380174 9962 raft_consensus.cc:3037] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:47.381799 9962 raft_consensus.cc:504] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49f68da0e5d4d00b56c3c0d5e62932b" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37957 } } peers { permanent_uuid: "b424b51e9d974b2aa86ea602c8d40edb" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44263 } } peers { permanent_uuid: "88a461eeb55b4f05ac2719ca97d06491" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46425 } }
I20260806 00:54:47.382297 9962 leader_election.cc:288] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [CANDIDATE]: Term 1 election: Requested vote from peers b49f68da0e5d4d00b56c3c0d5e62932b (127.27.77.194:37957), 88a461eeb55b4f05ac2719ca97d06491 (127.27.77.193:46425)
I20260806 00:54:47.382871 9504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a31b352a4654822b77bb587c5587572" candidate_uuid: "b424b51e9d974b2aa86ea602c8d40edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a461eeb55b4f05ac2719ca97d06491"
I20260806 00:54:47.382961 9688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a31b352a4654822b77bb587c5587572" candidate_uuid: "b424b51e9d974b2aa86ea602c8d40edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b49f68da0e5d4d00b56c3c0d5e62932b"
I20260806 00:54:47.383114 9688 raft_consensus.cc:3037] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:47.383087 9504 raft_consensus.cc:3037] T 8a31b352a4654822b77bb587c5587572 P 88a461eeb55b4f05ac2719ca97d06491 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:54:47.385459 9688 raft_consensus.cc:2454] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b424b51e9d974b2aa86ea602c8d40edb in term 1.
I20260806 00:54:47.385459 9504 raft_consensus.cc:2454] T 8a31b352a4654822b77bb587c5587572 P 88a461eeb55b4f05ac2719ca97d06491 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b424b51e9d974b2aa86ea602c8d40edb in term 1.
I20260806 00:54:47.386128 9775 leader_election.cc:302] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88a461eeb55b4f05ac2719ca97d06491, b424b51e9d974b2aa86ea602c8d40edb; no voters:
I20260806 00:54:47.386472 9962 raft_consensus.cc:2781] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:54:47.386895 9962 raft_consensus.cc:686] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [term 1 LEADER]: Becoming Leader. State: Replica: b424b51e9d974b2aa86ea602c8d40edb, State: Running, Role: LEADER
I20260806 00:54:47.387475 9962 consensus_queue.cc:237] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49f68da0e5d4d00b56c3c0d5e62932b" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37957 } } peers { permanent_uuid: "b424b51e9d974b2aa86ea602c8d40edb" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44263 } } peers { permanent_uuid: "88a461eeb55b4f05ac2719ca97d06491" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46425 } }
I20260806 00:54:47.392501 9316 catalog_manager.cc:5495] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb reported cstate change: term changed from 0 to 1, leader changed from <none> to b424b51e9d974b2aa86ea602c8d40edb (127.27.77.195). New cstate: current_term: 1 leader_uuid: "b424b51e9d974b2aa86ea602c8d40edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b49f68da0e5d4d00b56c3c0d5e62932b" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b424b51e9d974b2aa86ea602c8d40edb" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88a461eeb55b4f05ac2719ca97d06491" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46425 } health_report { overall_health: UNKNOWN } } }
I20260806 00:54:47.433434 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:47.429612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36972 (local address 127.27.77.195:44263)
0806 00:54:47.429758 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:54:47.429762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:54:47.429776 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:54:47.429861 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:54:47.429863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:54:47.429921 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:54:47.430077 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:54:47.430338 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.430858 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.431782 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:54:47.432046 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:54:47.432134 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:54:47.432493 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:54:47.432944 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:54:47.433053 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:54:47.433171 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 00:54:47.433229 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:54:47.439742 9688 raft_consensus.cc:1261] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b [term 1 FOLLOWER]: Refusing update from remote peer b424b51e9d974b2aa86ea602c8d40edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:47.439787 9504 raft_consensus.cc:1261] T 8a31b352a4654822b77bb587c5587572 P 88a461eeb55b4f05ac2719ca97d06491 [term 1 FOLLOWER]: Refusing update from remote peer b424b51e9d974b2aa86ea602c8d40edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:54:47.440634 9962 consensus_queue.cc:1035] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b49f68da0e5d4d00b56c3c0d5e62932b" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:47.440874 9964 consensus_queue.cc:1035] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a461eeb55b4f05ac2719ca97d06491" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:54:47.448397 9971 mvcc.cc:205] Tried to move back new op lower bound from 7315364607744524288 to 7315364607543119872. Current Snapshot: MvccSnapshot[applied={T|T < 7315364607744524288}]
I20260806 00:54:47.450618 9972 mvcc.cc:205] Tried to move back new op lower bound from 7315364607744524288 to 7315364607543119872. Current Snapshot: MvccSnapshot[applied={T|T < 7315364607744524288}]
I20260806 00:54:47.451905 9968 mvcc.cc:205] Tried to move back new op lower bound from 7315364607744524288 to 7315364607543119872. Current Snapshot: MvccSnapshot[applied={T|T < 7315364607744524288 or (T in {7315364607744524288})}]
I20260806 00:54:47.457898 9317 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:47.458161 9317 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:54:47.461046 9317 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dfcda7f8ed945f28d19b7d2b60676d3: Sending DeleteTablet for 3 replicas of tablet 8a31b352a4654822b77bb587c5587572
I20260806 00:54:47.462049 9841 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a31b352a4654822b77bb587c5587572 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:47 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:36960
I20260806 00:54:47.462067 9664 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a31b352a4654822b77bb587c5587572 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:47 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:38300
I20260806 00:54:47.462067 9479 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a31b352a4654822b77bb587c5587572 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:54:47 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:55836
I20260806 00:54:47.462631 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 9401
I20260806 00:54:47.464874 9986 tablet_replica.cc:330] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b: stopping tablet replica
I20260806 00:54:47.465227 9986 raft_consensus.cc:2229] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:54:47.465577 9986 raft_consensus.cc:2258] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:47.468820 9986 ts_tablet_manager.cc:1905] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:47.470839 9987 tablet_replica.cc:330] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb: stopping tablet replica
I20260806 00:54:47.471207 9987 raft_consensus.cc:2229] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:54:47.471679 9987 raft_consensus.cc:2258] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:54:47.472536 9986 ts_tablet_manager.cc:1918] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:47.472714 9986 log.cc:1196] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/8a31b352a4654822b77bb587c5587572
I20260806 00:54:47.473152 9986 ts_tablet_manager.cc:1939] T 8a31b352a4654822b77bb587c5587572 P b49f68da0e5d4d00b56c3c0d5e62932b: Deleting consensus metadata
I20260806 00:54:47.473443 9987 ts_tablet_manager.cc:1905] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:54:47.474707 9299 catalog_manager.cc:4841] TS b49f68da0e5d4d00b56c3c0d5e62932b (127.27.77.194:37957): tablet 8a31b352a4654822b77bb587c5587572 (table test-table [id=0e935715e4b94f97b7de61b80625fcc4]) successfully deleted
I20260806 00:54:47.476354 9987 ts_tablet_manager.cc:1918] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:54:47.476470 9987 log.cc:1196] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/8a31b352a4654822b77bb587c5587572
I20260806 00:54:47.476774 9987 ts_tablet_manager.cc:1939] T 8a31b352a4654822b77bb587c5587572 P b424b51e9d974b2aa86ea602c8d40edb: Deleting consensus metadata
I20260806 00:54:47.477759 9301 catalog_manager.cc:4841] TS b424b51e9d974b2aa86ea602c8d40edb (127.27.77.195:44263): tablet 8a31b352a4654822b77bb587c5587572 (table test-table [id=0e935715e4b94f97b7de61b80625fcc4]) successfully deleted
W20260806 00:54:47.480605 9297 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:46425 (error 108)
I20260806 00:54:47.480906 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 9573
I20260806 00:54:47.494803 9946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:54:47.494369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:46425 (local address 127.0.0.1:55850)
0806 00:54:47.494565 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 00:54:47.494666 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260806 00:54:47.495132 9297 catalog_manager.cc:4584] TS 88a461eeb55b4f05ac2719ca97d06491 (127.27.77.193:46425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a31b352a4654822b77bb587c5587572: Network error: Client connection negotiation failed: client connection to 127.27.77.193:46425: connect: Connection refused (error 111)
I20260806 00:54:47.500231 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 9752
I20260806 00:54:47.526218 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 9276
2026-08-06T00:54:47Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (3734 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:54:47 dist-test-slave-sb2z krb5kdc[9993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:54:47 dist-test-slave-sb2z krb5kdc[9993](info): set up 2 sockets
Aug 06 00:54:47 dist-test-slave-sb2z krb5kdc[9993](info): commencing operation
krb5kdc: starting...
W20260806 00:54:48.741279 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:54:48 dist-test-slave-sb2z krb5kdc[9993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:54:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:54:48Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.27.77.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.27.77.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/rangeradmin_127.27.77.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/rangeradmin_127.27.77.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.27.77.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.27.77.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/rangerlookup_127.27.77.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/rangerlookup_127.27.77.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.77.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.27.77.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/HTTP_127.27.77.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/HTTP_127.27.77.212@KRBTEST.COM.keytab.
I20260806 00:54:48.924626 27959 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskxB88Ks/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/postgres -l logfile start
2026-08-06 00:54:51.600 UTC [10028] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 00:54:51.600 UTC [10028] LOG: listening on IPv4 address "127.27.77.212", port 49231
2026-08-06 00:54:51.604 UTC [10028] LOG: listening on Unix socket "/tmp/.s.PGSQL.49231"
2026-08-06 00:54:51.641 UTC [10034] LOG: database system was shut down at 2026-08-06 00:54:50 UTC
2026-08-06 00:54:51.646 UTC [10028] LOG: database system is ready to accept connections
I20260806 00:54:52.755571 27959 mini_postgres.cc:96] Postgres bound to 49231
2026-08-06 00:54:52.764 UTC [10045] FATAL: database "slave" does not exist
127.27.77.212:49231 - accepting connections
I20260806 00:54:52.765316 27959 mini_ranger.cc:161] Starting Ranger...
I20260806 00:54:52.784579 27959 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 00:54:53.313405 27959 mini_ranger.cc:87] Created ranger Postgres database
I20260806 00:54:53.313519 27959 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-admin
2026-08-06 00:54:53,698 [I] DB FLAVOR :POSTGRES
2026-08-06 00:54:53,699 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 00:54:53,699 [I] Checking connection..
2026-08-06 00:54:53,699 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 00:54:53,987 [I] Checking connection passed.
2026-08-06 00:54:53,988 [I] --------- Verifying version history table ---------
2026-08-06 00:54:53,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:54:54,266 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 00:54:54,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:54:54,516 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 00:54:54,516 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 00:54:54,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 00:54:54.784 UTC [10136] WARNING: there is no transaction in progress
2026-08-06 00:54:54,792 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 00:54:54,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:54:55,053 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 00:54:55,054 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 00:54:55,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 00:54:55,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 00:54:55,622 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sb2z.c.gcp-upstream.internal','N') ;"
2026-08-06 00:54:55,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 00:54:56,133 [I] Table x_portal_user does not exist in database ranger
2026-08-06 00:54:56,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 00:54:56,406 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 00:54:56,407 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 00:54:56,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 00:54:56,926 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 00:54:56,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 00:54:57.971 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:57.989 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:58.008 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:58.025 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:58.043 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:58.062 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:58.119 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:58.126 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:58.142 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:58.152 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:58.164 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:58.178 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:58.188 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:58.195 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:58.202 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:58.210 UTC [10274] WARNING: there is no transaction in progress
2026-08-06 00:54:58,908 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 00:54:58,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 00:54:59,190 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 00:54:59,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 00:54:59,452 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 00:54:59,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 00:54:59,719 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 00:54:59,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 00:54:59,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 00:55:00,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sb2z.c.gcp-upstream.internal';"
2026-08-06 00:55:00,506 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 00:55:00,507 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 00:55:00,792 [I] DB_PATCHES have already been applied
I20260806 00:55:00.798355 27959 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-admin:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskxB88Ks/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 00:55:00.798426 27959 mini_ranger.cc:193] Using host: 127.27.77.212
I20260806 00:55:00.803498 27959 mini_ranger.cc:227] Ranger admin URL: http://127.27.77.212:46837
Aug 06, 2026 12:55:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:55:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:55:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 12:55:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 12:55:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 12:55:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 12:55:01 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 00:55:02.162178 27959 mini_ranger.cc:160] Time spent starting Ranger: real 9.397s user 0.000s sys 0.014s
Aug 06, 2026 12:55:23 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 12:55:23 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 06, 2026 12:55:23 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 12:55:23 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 12:55:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 12:55:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 12:55:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 12:55:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 12:55:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 12:55:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 12:55:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 12:55:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 12:55:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 12:55:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 12:55:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 12:55:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 12:55:23 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 12:55:24 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 00:55:25.483332 27959 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.27.77.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.27.77.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/rangerkms_127.27.77.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/rangerkms_127.27.77.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.77.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.27.77.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.77.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/HTTP_127.27.77.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.77.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/HTTP_127.27.77.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 00:55:25 dist-test-slave-sb2z krb5kdc[9993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977725, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 00:55:25.592018 27959 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 00:55:25.603816 27959 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 00:55:26.642225 27959 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 00:55:26.642310 27959 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-kms
I20260806 00:55:26.642412 27959 mini_ranger_kms.cc:227] Using postgres at 127.27.77.212:49231
2026-08-06 00:55:27,238 [I] DB FLAVOR :POSTGRES
2026-08-06 00:55:27,239 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 00:55:27,239 [I] Checking connection
2026-08-06 00:55:27,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 00:55:27,520 [I] connection success
2026-08-06 00:55:27,520 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 00:55:27,520 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 00:55:27,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 00:55:27,816 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 00:55:27,816 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 00:55:27,816 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 00:55:27,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:49231/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 00:55:28,114 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 00:55:28.638404 27959 mini_ranger_kms.cc:310] Created kms service
I20260806 00:55:28.835970 27959 mini_ranger_kms.cc:326] Created kudu user
I20260806 00:55:28.886889 27959 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 00:55:28 dist-test-slave-sb2z krb5kdc[9993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977725, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:55:29.349315 27959 mini_ranger_kms.cc:384] Added ranger policy
I20260806 00:55:29.349720 27959 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-kms:/tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 00:55:29.349774 27959 mini_ranger_kms.cc:240] Using host: 127.27.77.212
I20260806 00:55:29.355228 27959 mini_ranger_kms.cc:276] Ranger KMS PID: 10500
I20260806 00:55:29.355391 27959 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.27.77.212:44063
Aug 06, 2026 12:55:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:55:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:55:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 12:55:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 12:55:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 12:55:31 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 00:55:31 dist-test-slave-sb2z krb5kdc[9993](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785977731, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.77.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:55:32.045017 27959 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.453s user 0.036s sys 0.300s
I20260806 00:55:32.045169 27959 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 00:55:32.045282 27959 mini_ranger_kms.cc:401] 127.27.77.212:44063/kms/v1/keys
Aug 06 00:55:32 dist-test-slave-sb2z krb5kdc[9993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977731, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.77.212@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
Aug 06 00:55:32 dist-test-slave-sb2z krb5kdc[9993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977731, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.77.212@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
Aug 06 00:55:34 dist-test-slave-sb2z krb5kdc[9993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:55:35.030943 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:33531
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:44095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:33531
--ranger_config_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.77.212:44063/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:55:35.152475 10571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:55:35.152751 10571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:55:35.152791 10571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:55:35.156805 10571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:55:35.156867 10571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:55:35.156885 10571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:55:35.156903 10571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:55:35.156919 10571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:55:35.162353 10571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:44095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/wal
--ranger_kms_url=127.27.77.212:44063/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:33531
--ranger_config_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:33531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:55:35.163360 10571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:55:35.164527 10571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:55:35.175165 10571 server_base.cc:1034] running on GCE node
W20260806 00:55:35.175108 10581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:55:35.175123 10578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:55:35.175104 10579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:55:35.176024 10571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:55:35.177193 10571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:55:35.178391 10571 hybrid_clock.cc:648] HybridClock initialized: now 1785977735178382 us; error 44 us; skew 500 ppm
Aug 06 00:55:35 dist-test-slave-sb2z krb5kdc[9993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977735, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:55:35.181386 10571 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:55:35.182299 10571 webserver.cc:460] Webserver started at http://127.27.77.254:34351/ using document root <none> and password file <none>
I20260806 00:55:35.182862 10571 fs_manager.cc:359] Metadata directory not provided
I20260806 00:55:35.182955 10571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:55:35.183213 10571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:55:35 dist-test-slave-sb2z krb5kdc[9993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977735, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:55:35.241195 10571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "9a0b3aed7f3d4d2d881eb034680b8f25"
format_stamp: "Formatted at 2026-08-06 00:55:35 on dist-test-slave-sb2z"
server_key: "e3eefbeb608dd6798ee33de39a292705"
server_key_iv: "c6c423510bcc61c3f8432dca6095227c"
server_key_version: "kuduclusterkey@0"
I20260806 00:55:35.241978 10571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "9a0b3aed7f3d4d2d881eb034680b8f25"
format_stamp: "Formatted at 2026-08-06 00:55:35 on dist-test-slave-sb2z"
server_key: "e3eefbeb608dd6798ee33de39a292705"
server_key_iv: "c6c423510bcc61c3f8432dca6095227c"
server_key_version: "kuduclusterkey@0"
I20260806 00:55:35.246423 10571 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 00:55:35.264513 10588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:55:35.266222 10571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:55:35.266382 10571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "9a0b3aed7f3d4d2d881eb034680b8f25"
format_stamp: "Formatted at 2026-08-06 00:55:35 on dist-test-slave-sb2z"
server_key: "e3eefbeb608dd6798ee33de39a292705"
server_key_iv: "c6c423510bcc61c3f8432dca6095227c"
server_key_version: "kuduclusterkey@0"
I20260806 00:55:35.266530 10571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:55:35.305532 10571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:55:35.309242 10571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:55:35.309458 10571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:55:35.318982 10571 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:33531
I20260806 00:55:35.320382 10571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:55:35.320750 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 10571
I20260806 00:55:35.321133 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:55:35.321548 10652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:33531 every 8 connection(s)
I20260806 00:55:35.321887 27959 external_mini_cluster.cc:1428] Setting key c9c4d1c14aa7fc53a4c917c9b0030d2f
I20260806 00:55:35.325359 10654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:55:35.332932 10654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25: Bootstrap starting.
I20260806 00:55:35.336076 10654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25: Neither blocks nor log segments found. Creating new log.
I20260806 00:55:35.337132 10654 log.cc:824] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25: Log is configured to *not* fsync() on all Append() calls
I20260806 00:55:35.339860 10654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25: No bootstrap required, opened a new log
I20260806 00:55:35.344933 10654 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0b3aed7f3d4d2d881eb034680b8f25" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33531 } }
I20260806 00:55:35.345261 10654 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:55:35.345359 10654 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0b3aed7f3d4d2d881eb034680b8f25, State: Initialized, Role: FOLLOWER
I20260806 00:55:35.345898 10654 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0b3aed7f3d4d2d881eb034680b8f25" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33531 } }
I20260806 00:55:35.346072 10654 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:55:35.346171 10654 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:55:35.346297 10654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 00:55:35 dist-test-slave-sb2z krb5kdc[9993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:55:35.347702 10654 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0b3aed7f3d4d2d881eb034680b8f25" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33531 } }
I20260806 00:55:35.348165 10654 leader_election.cc:302] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a0b3aed7f3d4d2d881eb034680b8f25; no voters:
I20260806 00:55:35.348642 10654 leader_election.cc:288] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:55:35.349668 10661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:55:35.349982 10661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [term 1 LEADER]: Becoming Leader. State: Replica: 9a0b3aed7f3d4d2d881eb034680b8f25, State: Running, Role: LEADER
I20260806 00:55:35.350420 10661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0b3aed7f3d4d2d881eb034680b8f25" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33531 } }
I20260806 00:55:35.350986 10654 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:55:35.351953 10661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a0b3aed7f3d4d2d881eb034680b8f25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0b3aed7f3d4d2d881eb034680b8f25" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33531 } } }
I20260806 00:55:35.352181 10661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [sys.catalog]: This master's current role is: LEADER
I20260806 00:55:35.352445 10661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a0b3aed7f3d4d2d881eb034680b8f25. Latest consensus state: current_term: 1 leader_uuid: "9a0b3aed7f3d4d2d881eb034680b8f25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0b3aed7f3d4d2d881eb034680b8f25" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33531 } } }
I20260806 00:55:35.353308 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:35.324683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39620 (local address 127.27.77.254:33531)
0806 00:55:35.325372 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0806 00:55:35.325392 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:55:35.337744 (+ 12352us) server_negotiation.cc:407] Connection header received
0806 00:55:35.338737 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:55:35.338764 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:55:35.339239 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:55:35.339698 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:55:35.340729 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:35.342095 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:35.343604 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:35.344221 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:35.344420 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:35.347434 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:55:35.347440 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:55:35.347456 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:55:35.347489 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:55:35.349460 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:55:35.350107 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:55:35.350115 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:55:35.350123 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:55:35.350264 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:55:35.350641 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:55:35.350644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:55:35.350646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:55:35.351179 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:55:35.351345 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 00:55:35.351883 (+ 538us) server_negotiation.cc:299] Negotiation successful
0806 00:55:35.352195 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":164,"threads_started":1}
I20260806 00:55:35.353613 10654 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 00:55:35.352586 10661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25 [sys.catalog]: This master's current role is: LEADER
I20260806 00:55:35.358758 10673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:55:35.364715 10673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:55:35.372677 10673 catalog_manager.cc:1347] Generated new cluster ID: 5ef2ffe8a51c4db580f5636b9b52e33f
I20260806 00:55:35.372756 10673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:55:35.413393 10673 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:55:35.414906 10673 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:55:35.425004 10673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25: Generated new TSK 0
I20260806 00:55:35.426033 10673 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:55:36.151656 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.77.212:44063/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33531
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:44095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:55:36.307771 10709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:55:36.308156 10709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:55:36.308255 10709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:55:36.313105 10709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:55:36.313198 10709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:55:36.313289 10709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:55:36.319290 10709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:44095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.27.77.212:44063/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:55:36.321099 10709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:55:36.322397 10709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:55:36.335323 10717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:55:36.335170 10723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:55:36.335150 10719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:55:36.335671 10709 server_base.cc:1034] running on GCE node
I20260806 00:55:36.336161 10709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:55:36.337035 10709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:55:36.338318 10709 hybrid_clock.cc:648] HybridClock initialized: now 1785977736338287 us; error 47 us; skew 500 ppm
Aug 06 00:55:36 dist-test-slave-sb2z krb5kdc[9993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977736, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:55:36.342315 10709 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:55:36.343636 10709 webserver.cc:460] Webserver started at http://127.27.77.193:39359/ using document root <none> and password file <none>
I20260806 00:55:36.344466 10709 fs_manager.cc:359] Metadata directory not provided
I20260806 00:55:36.344635 10709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:55:36.344977 10709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:55:36 dist-test-slave-sb2z krb5kdc[9993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977736, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:55:36.363395 10709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "4d813d88021f47b98488a9da2854033f"
format_stamp: "Formatted at 2026-08-06 00:55:36 on dist-test-slave-sb2z"
server_key: "8843bbcce96f4cad0c7d9bc0bae46a85"
server_key_iv: "b20d9c73ca4e6074b8c50a53cd556e48"
server_key_version: "kuduclusterkey@0"
I20260806 00:55:36.364305 10709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "4d813d88021f47b98488a9da2854033f"
format_stamp: "Formatted at 2026-08-06 00:55:36 on dist-test-slave-sb2z"
server_key: "8843bbcce96f4cad0c7d9bc0bae46a85"
server_key_iv: "b20d9c73ca4e6074b8c50a53cd556e48"
server_key_version: "kuduclusterkey@0"
I20260806 00:55:36.369848 10709 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 00:55:36.389197 10732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:55:36.390951 10709 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:55:36.391089 10709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "4d813d88021f47b98488a9da2854033f"
format_stamp: "Formatted at 2026-08-06 00:55:36 on dist-test-slave-sb2z"
server_key: "8843bbcce96f4cad0c7d9bc0bae46a85"
server_key_iv: "b20d9c73ca4e6074b8c50a53cd556e48"
server_key_version: "kuduclusterkey@0"
I20260806 00:55:36.391230 10709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:55:36.440121 10709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:55:36.444203 10709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:55:36.444494 10709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:55:36.445328 10709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:55:36.446358 10709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:55:36.446434 10709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:55:36.446532 10709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:55:36.446682 10709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:55:36.459627 10709 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:40095
I20260806 00:55:36.460749 10709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:55:36.467295 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 10709
I20260806 00:55:36.467388 10879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:40095 every 8 connection(s)
I20260806 00:55:36.467442 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:55:36.467818 27959 external_mini_cluster.cc:1428] Setting key a26991e6c34566872657b1ea90ce40af
Aug 06 00:55:36 dist-test-slave-sb2z krb5kdc[9993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977736, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
I20260806 00:55:36.486145 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:36.471188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:57739 (local address 127.27.77.254:33531)
0806 00:55:36.472960 (+ 1772us) server_negotiation.cc:206] Beginning negotiation
0806 00:55:36.472965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:55:36.472982 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:55:36.473028 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:55:36.473031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:55:36.473082 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:55:36.474759 (+ 1677us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:55:36.474882 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:36.475766 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:36.477487 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:36.477842 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:36.477928 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:36.481370 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:55:36.481375 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:55:36.481377 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:55:36.481408 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:55:36.483449 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:55:36.484204 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:55:36.484208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:55:36.484210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:55:36.484310 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:55:36.484741 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:55:36.484745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:55:36.484747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:55:36.484983 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:55:36.485056 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:55:36.485798 (+ 742us) server_negotiation.cc:299] Negotiation successful
0806 00:55:36.485939 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1673,"thread_start_us":88,"threads_started":1}
I20260806 00:55:36.487903 10882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:36.469964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:33531 (local address 127.27.77.193:57739)
0806 00:55:36.470718 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0806 00:55:36.470795 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 00:55:36.472348 (+ 1553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:55:36.473427 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:55:36.473439 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:55:36.473865 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:55:36.474589 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:36.474613 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:36.475917 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:36.475921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:36.477324 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:36.477337 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:36.478174 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:36.478178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:36.478396 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:36.479128 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:55:36.479151 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:55:36.481138 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:55:36.483646 (+ 2508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:55:36.483653 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:55:36.483674 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:55:36.484057 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:55:36.484428 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:55:36.484431 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:55:36.484433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:55:36.484625 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:55:36.485158 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:55:36.485165 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:55:36.485504 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 00:55:36.486339 (+ 835us) client_negotiation.cc:240] Negotiation successful
0806 00:55:36.486722 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":198,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
I20260806 00:55:36.489553 10880 heartbeater.cc:347] Connected to a master server at 127.27.77.254:33531
I20260806 00:55:36.489835 10880 heartbeater.cc:464] Registering TS with master...
I20260806 00:55:36.490398 10880 heartbeater.cc:511] Master 127.27.77.254:33531 requested a full tablet report, sending...
I20260806 00:55:36.492257 10608 ts_manager.cc:194] Registered new tserver with Master: 4d813d88021f47b98488a9da2854033f (127.27.77.193:40095)
I20260806 00:55:36.493844 10608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:57739
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:55:36.511139 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.77.212:44063/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33531
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:44095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:55:36 dist-test-slave-sb2z krb5kdc[9993](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785977736, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 00:55:36.687897 10886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:55:36.688218 10886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:55:36.688267 10886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:55:36.693290 10886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:55:36.693451 10886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:55:36.693572 10886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:55:36.700698 10886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:44095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.27.77.212:44063/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:55:36.701999 10886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:55:36.703096 10886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:55:36.715011 10903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:55:36.715272 10895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:55:36.715058 10892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:55:36.715907 10886 server_base.cc:1034] running on GCE node
I20260806 00:55:36.716359 10886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:55:36.717126 10886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:55:36.718333 10886 hybrid_clock.cc:648] HybridClock initialized: now 1785977736718301 us; error 54 us; skew 500 ppm
Aug 06 00:55:36 dist-test-slave-sb2z krb5kdc[9993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977736, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:55:36.722404 10886 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:55:36.723783 10886 webserver.cc:460] Webserver started at http://127.27.77.194:39183/ using document root <none> and password file <none>
I20260806 00:55:36.724604 10886 fs_manager.cc:359] Metadata directory not provided
I20260806 00:55:36.724721 10886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:55:36.724957 10886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:55:36 dist-test-slave-sb2z krb5kdc[9993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977736, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:55:36.741894 10886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "6f4786a3c2c84cd8889e9f9629af0bec"
format_stamp: "Formatted at 2026-08-06 00:55:36 on dist-test-slave-sb2z"
server_key: "9ae1e54309d8593b0200bdf76eb73290"
server_key_iv: "b996776171246e6159c1cb6b4a5eebba"
server_key_version: "kuduclusterkey@0"
I20260806 00:55:36.742940 10886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "6f4786a3c2c84cd8889e9f9629af0bec"
format_stamp: "Formatted at 2026-08-06 00:55:36 on dist-test-slave-sb2z"
server_key: "9ae1e54309d8593b0200bdf76eb73290"
server_key_iv: "b996776171246e6159c1cb6b4a5eebba"
server_key_version: "kuduclusterkey@0"
I20260806 00:55:36.751148 10886 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.000s
I20260806 00:55:36.773411 10913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:55:36.775244 10886 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260806 00:55:36.775471 10886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "6f4786a3c2c84cd8889e9f9629af0bec"
format_stamp: "Formatted at 2026-08-06 00:55:36 on dist-test-slave-sb2z"
server_key: "9ae1e54309d8593b0200bdf76eb73290"
server_key_iv: "b996776171246e6159c1cb6b4a5eebba"
server_key_version: "kuduclusterkey@0"
I20260806 00:55:36.775751 10886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:55:36.800206 10886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:55:36.804417 10886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:55:36.804777 10886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:55:36.805723 10886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:55:36.806994 10886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:55:36.807109 10886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:55:36.807214 10886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:55:36.807296 10886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:55:36.833077 10886 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:33323
I20260806 00:55:36.834810 10886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:55:36.837791 11052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:33323 every 8 connection(s)
I20260806 00:55:36.844112 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 10886
I20260806 00:55:36.844292 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:55:36.844667 27959 external_mini_cluster.cc:1428] Setting key b0cbcf6923f27311282a97dd449d18ba
Aug 06 00:55:36 dist-test-slave-sb2z krb5kdc[9993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977736, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:55:36.864687 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:36.842094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:49089 (local address 127.27.77.254:33531)
0806 00:55:36.842239 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 00:55:36.842243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:55:36.843298 (+ 1055us) server_negotiation.cc:407] Connection header received
0806 00:55:36.846107 (+ 2809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:55:36.846110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:55:36.846168 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:55:36.846222 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:55:36.850915 (+ 4693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:36.851777 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:36.853844 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:36.854235 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:36.854325 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:36.857948 (+ 3623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:55:36.857953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:55:36.857956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:55:36.857990 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:55:36.859891 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:55:36.862407 (+ 2516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:55:36.862411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:55:36.862413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:55:36.862531 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:55:36.863050 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:55:36.863053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:55:36.863055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:55:36.863317 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:55:36.863399 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 00:55:36.864243 (+ 844us) server_negotiation.cc:299] Negotiation successful
0806 00:55:36.864416 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:55:36.866850 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:36.840746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:33531 (local address 127.27.77.194:49089)
0806 00:55:36.843097 (+ 2351us) negotiation.cc:107] Waiting for socket to connect
0806 00:55:36.843162 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 00:55:36.845823 (+ 2661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:55:36.849464 (+ 3641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:55:36.849476 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:55:36.849910 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:55:36.850695 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:36.850721 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:36.851978 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:36.851981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:36.853671 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:36.853686 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:36.854574 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:36.854577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:36.854817 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:36.855772 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:55:36.855796 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:55:36.857676 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:55:36.860093 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:55:36.860100 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:55:36.860123 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:55:36.860530 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:55:36.862686 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:55:36.862689 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:55:36.862691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:55:36.862921 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:55:36.863494 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:55:36.863502 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:55:36.863930 (+ 428us) client_negotiation.cc:769] Sending connection context
0806 00:55:36.865228 (+ 1298us) client_negotiation.cc:240] Negotiation successful
0806 00:55:36.865665 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2102,"thread_start_us":268,"threads_started":1}
I20260806 00:55:36.868563 11053 heartbeater.cc:347] Connected to a master server at 127.27.77.254:33531
I20260806 00:55:36.868844 11053 heartbeater.cc:464] Registering TS with master...
I20260806 00:55:36.869449 11053 heartbeater.cc:511] Master 127.27.77.254:33531 requested a full tablet report, sending...
I20260806 00:55:36.870723 10608 ts_manager.cc:194] Registered new tserver with Master: 6f4786a3c2c84cd8889e9f9629af0bec (127.27.77.194:33323)
I20260806 00:55:36.871568 10608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:49089
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:55:36.919499 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.77.212:44063/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33531
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:44095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:55:37.092000 11059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:55:37.092451 11059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:55:37.092571 11059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:55:37.098613 11059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:55:37.098769 11059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:55:37.098898 11059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:55:37.106429 11059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:44095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.27.77.212:44063/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:55:37.108176 11059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:55:37.109561 11059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:55:37.122993 11065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:55:37.124807 11059 server_base.cc:1034] running on GCE node
W20260806 00:55:37.127732 11064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:55:37.127965 11067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:55:37.128626 11059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:55:37.129480 11059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:55:37.143731 11059 hybrid_clock.cc:648] HybridClock initialized: now 1785977737143626 us; error 118 us; skew 500 ppm
Aug 06 00:55:37 dist-test-slave-sb2z krb5kdc[9993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977737, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:55:37.147825 11059 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:55:37.149196 11059 webserver.cc:460] Webserver started at http://127.27.77.195:42489/ using document root <none> and password file <none>
I20260806 00:55:37.150015 11059 fs_manager.cc:359] Metadata directory not provided
I20260806 00:55:37.150122 11059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:55:37.150394 11059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:55:37 dist-test-slave-sb2z krb5kdc[9993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977737, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:55:37.178941 11059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "c9a39d239311471cad59a156ddca6fd6"
format_stamp: "Formatted at 2026-08-06 00:55:37 on dist-test-slave-sb2z"
server_key: "463bc94369995014db56715c237c272d"
server_key_iv: "cf9990c5d22ea55c186238d4afe8c1e1"
server_key_version: "kuduclusterkey@0"
I20260806 00:55:37.181043 11059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "c9a39d239311471cad59a156ddca6fd6"
format_stamp: "Formatted at 2026-08-06 00:55:37 on dist-test-slave-sb2z"
server_key: "463bc94369995014db56715c237c272d"
server_key_iv: "cf9990c5d22ea55c186238d4afe8c1e1"
server_key_version: "kuduclusterkey@0"
I20260806 00:55:37.201460 11059 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.004s sys 0.000s
I20260806 00:55:37.236593 11079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:55:37.253587 11059 fs_manager.cc:725] Time spent opening block manager: real 0.033s user 0.003s sys 0.000s
I20260806 00:55:37.255839 11059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "c9a39d239311471cad59a156ddca6fd6"
format_stamp: "Formatted at 2026-08-06 00:55:37 on dist-test-slave-sb2z"
server_key: "463bc94369995014db56715c237c272d"
server_key_iv: "cf9990c5d22ea55c186238d4afe8c1e1"
server_key_version: "kuduclusterkey@0"
I20260806 00:55:37.256158 11059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 06 00:55:37 dist-test-slave-sb2z krb5kdc[9993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977736, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:55:37.361611 11059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:55:37.371132 11059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:55:37.371434 11059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:55:37.372293 11059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:55:37.373540 11059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:55:37.373648 11059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:55:37.373750 11059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:55:37.373813 11059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:55:37.392998 11059 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:44125
I20260806 00:55:37.393148 11223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:44125 every 8 connection(s)
I20260806 00:55:37.394313 11059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:55:37.400674 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 11059
I20260806 00:55:37.400806 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:55:37.401113 27959 external_mini_cluster.cc:1428] Setting key 6c11e36943b37a3ef17c5b7609560d07
Aug 06 00:55:37 dist-test-slave-sb2z krb5kdc[9993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977737, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:55:37.417600 11228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.398892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:44323 (local address 127.27.77.254:33531)
0806 00:55:37.401038 (+ 2146us) server_negotiation.cc:206] Beginning negotiation
0806 00:55:37.401043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:55:37.401063 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:55:37.402134 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:55:37.402137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:55:37.402194 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:55:37.403348 (+ 1154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:55:37.404990 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.405845 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.407585 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.407940 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.408017 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:37.411219 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:55:37.411223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:55:37.411226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:55:37.411253 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:55:37.413176 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:55:37.414458 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:55:37.415149 (+ 691us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:55:37.415150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:55:37.415251 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:55:37.415714 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:55:37.415718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:55:37.415720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:55:37.416408 (+ 688us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:55:37.416517 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:55:37.417256 (+ 739us) server_negotiation.cc:299] Negotiation successful
0806 00:55:37.417398 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2049,"thread_start_us":104,"threads_started":1}
I20260806 00:55:37.419554 11226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.397766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:33531 (local address 127.27.77.195:44323)
0806 00:55:37.399509 (+ 1743us) negotiation.cc:107] Waiting for socket to connect
0806 00:55:37.399596 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0806 00:55:37.401101 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:55:37.403751 (+ 2650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:55:37.403760 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:55:37.404139 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:55:37.404823 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:37.404841 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.405998 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.406001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:37.407405 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:37.407417 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.408272 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.408275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:37.408491 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:37.409223 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:55:37.409243 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:55:37.411024 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:55:37.413323 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:55:37.413328 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:55:37.413346 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:55:37.414341 (+ 995us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:55:37.415372 (+ 1031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:55:37.415375 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:55:37.415377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:55:37.415605 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:55:37.416595 (+ 990us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:55:37.416602 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:55:37.416948 (+ 346us) client_negotiation.cc:769] Sending connection context
0806 00:55:37.418106 (+ 1158us) client_negotiation.cc:240] Negotiation successful
0806 00:55:37.418474 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1491,"thread_start_us":123,"threads_started":1}
I20260806 00:55:37.421173 11224 heartbeater.cc:347] Connected to a master server at 127.27.77.254:33531
I20260806 00:55:37.421428 11224 heartbeater.cc:464] Registering TS with master...
I20260806 00:55:37.422051 11224 heartbeater.cc:511] Master 127.27.77.254:33531 requested a full tablet report, sending...
I20260806 00:55:37.423269 10608 ts_manager.cc:194] Registered new tserver with Master: c9a39d239311471cad59a156ddca6fd6 (127.27.77.195:44125)
I20260806 00:55:37.424064 10608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:44323
I20260806 00:55:37.427835 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:55:37.440907 11228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.430182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39622 (local address 127.27.77.254:33531)
0806 00:55:37.430299 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 00:55:37.430304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:55:37.430900 (+ 596us) server_negotiation.cc:407] Connection header received
0806 00:55:37.431111 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:55:37.431114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:55:37.431166 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:55:37.431218 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:55:37.432496 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.433319 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.434541 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.434912 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.434995 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:37.437015 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:55:37.437019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:55:37.437022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:55:37.437055 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:55:37.438787 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:55:37.439318 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:55:37.439322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:55:37.439324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:55:37.439419 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:55:37.439805 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:55:37.439809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:55:37.439811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:55:37.440050 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:55:37.440169 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 00:55:37.440590 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 00:55:37.440729 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 00:55:37.445124 10608 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:55:37.449839 10608 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:55:37.477370 11240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.465079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:44125 (local address 127.0.0.1:60784)
0806 00:55:37.467341 (+ 2262us) negotiation.cc:107] Waiting for socket to connect
0806 00:55:37.467352 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:55:37.467425 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:55:37.469361 (+ 1936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:55:37.469364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:55:37.469382 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:55:37.469475 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:37.469481 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.470977 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.470981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:37.475375 (+ 4394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:37.475386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.476794 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.476798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:37.476912 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:37.476954 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 00:55:37.477072 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 00:55:37.477211 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2213,"thread_start_us":118,"threads_started":1}
I20260806 00:55:37.478040 11248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.465557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60784 (local address 127.27.77.195:44125)
0806 00:55:37.466931 (+ 1374us) server_negotiation.cc:206] Beginning negotiation
0806 00:55:37.466952 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:55:37.467545 (+ 593us) server_negotiation.cc:407] Connection header received
0806 00:55:37.467630 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:55:37.467637 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:55:37.467849 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:55:37.468046 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:55:37.469614 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.470838 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.475573 (+ 4735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.476431 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.476557 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:37.476723 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 00:55:37.477782 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0806 00:55:37.477897 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1265,"thread_start_us":170,"threads_started":1}
I20260806 00:55:37.484627 11129 tablet_service.cc:1467] Processing CreateTablet for tablet 4b5950effb674f69a18eb9a674411875 (DEFAULT_TABLE table=test-table [id=faac6df70d334869b93da346cdcfa88a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:55:37.485980 11129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b5950effb674f69a18eb9a674411875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:55:37.480010 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.462653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:40095 (local address 127.0.0.1:40862)
0806 00:55:37.466342 (+ 3689us) negotiation.cc:107] Waiting for socket to connect
0806 00:55:37.466404 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 00:55:37.466556 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:55:37.468969 (+ 2413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:55:37.468976 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:55:37.469001 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:55:37.469242 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:37.469250 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.473773 (+ 4523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.473777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:37.474944 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:37.474957 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.479591 (+ 4634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.479598 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:37.479691 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:37.479715 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 00:55:37.479795 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 00:55:37.479876 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3484,"thread_start_us":112,"threads_started":1}
I20260806 00:55:37.496734 10880 heartbeater.cc:503] Master 127.27.77.254:33531 was elected leader, sending a full tablet report...
I20260806 00:55:37.497370 11241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.463513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40862 (local address 127.27.77.193:40095)
0806 00:55:37.468351 (+ 4838us) server_negotiation.cc:206] Beginning negotiation
0806 00:55:37.468358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:55:37.468382 (+ 24us) server_negotiation.cc:407] Connection header received
0806 00:55:37.468463 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:55:37.468468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:55:37.468653 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:55:37.468796 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:55:37.471454 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.473627 (+ 2173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.478294 (+ 4667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.479228 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.479343 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:37.479497 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 00:55:37.497034 (+ 17537us) server_negotiation.cc:299] Negotiation successful
0806 00:55:37.497187 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4726,"thread_start_us":139,"threads_started":1}
I20260806 00:55:37.500895 10792 tablet_service.cc:1467] Processing CreateTablet for tablet 4b5950effb674f69a18eb9a674411875 (DEFAULT_TABLE table=test-table [id=faac6df70d334869b93da346cdcfa88a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:55:37.502137 10792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b5950effb674f69a18eb9a674411875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:55:37.515250 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.464393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:33323 (local address 127.0.0.1:35336)
0806 00:55:37.467092 (+ 2699us) negotiation.cc:107] Waiting for socket to connect
0806 00:55:37.467105 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:55:37.467187 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:55:37.492187 (+ 25000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:55:37.492192 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:55:37.492222 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:55:37.492352 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:37.492362 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.512093 (+ 19731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.512097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:37.513386 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:37.513403 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.514774 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.514778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:37.514885 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:37.514919 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 00:55:37.514981 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 00:55:37.515077 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2638,"thread_start_us":119,"threads_started":1}
I20260806 00:55:37.516372 11243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.464571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35336 (local address 127.27.77.194:33323)
0806 00:55:37.480279 (+ 15708us) server_negotiation.cc:206] Beginning negotiation
0806 00:55:37.480286 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:55:37.480315 (+ 29us) server_negotiation.cc:407] Connection header received
0806 00:55:37.480410 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:55:37.480416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:55:37.480674 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:55:37.492578 (+ 11904us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:55:37.492741 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.493916 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.513626 (+ 19710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.514597 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.515856 (+ 1259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:37.516042 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 00:55:37.516108 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 00:55:37.516218 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15571,"thread_start_us":108,"threads_started":1}
I20260806 00:55:37.519510 11249 tablet_bootstrap.cc:492] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f: Bootstrap starting.
I20260806 00:55:37.521351 10970 tablet_service.cc:1467] Processing CreateTablet for tablet 4b5950effb674f69a18eb9a674411875 (DEFAULT_TABLE table=test-table [id=faac6df70d334869b93da346cdcfa88a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:55:37.522334 11250 tablet_bootstrap.cc:492] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6: Bootstrap starting.
I20260806 00:55:37.522610 10970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b5950effb674f69a18eb9a674411875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:55:37.523283 11249 tablet_bootstrap.cc:654] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f: Neither blocks nor log segments found. Creating new log.
I20260806 00:55:37.526346 11250 tablet_bootstrap.cc:654] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6: Neither blocks nor log segments found. Creating new log.
I20260806 00:55:37.527410 11249 log.cc:824] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f: Log is configured to *not* fsync() on all Append() calls
I20260806 00:55:37.528869 11250 log.cc:824] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6: Log is configured to *not* fsync() on all Append() calls
I20260806 00:55:37.531548 11249 tablet_bootstrap.cc:492] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f: No bootstrap required, opened a new log
I20260806 00:55:37.531898 11249 ts_tablet_manager.cc:1397] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.005s
I20260806 00:55:37.536084 11250 tablet_bootstrap.cc:492] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6: No bootstrap required, opened a new log
I20260806 00:55:37.536366 11250 ts_tablet_manager.cc:1397] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260806 00:55:37.537557 11249 raft_consensus.cc:348] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d813d88021f47b98488a9da2854033f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40095 } } peers { permanent_uuid: "6f4786a3c2c84cd8889e9f9629af0bec" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 33323 } } peers { permanent_uuid: "c9a39d239311471cad59a156ddca6fd6" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44125 } }
I20260806 00:55:37.538002 11249 raft_consensus.cc:374] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:55:37.538120 11249 raft_consensus.cc:729] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d813d88021f47b98488a9da2854033f, State: Initialized, Role: FOLLOWER
I20260806 00:55:37.538771 11249 consensus_queue.cc:260] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d813d88021f47b98488a9da2854033f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40095 } } peers { permanent_uuid: "6f4786a3c2c84cd8889e9f9629af0bec" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 33323 } } peers { permanent_uuid: "c9a39d239311471cad59a156ddca6fd6" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44125 } }
I20260806 00:55:37.540175 11249 ts_tablet_manager.cc:1428] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260806 00:55:37.541733 11250 raft_consensus.cc:348] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d813d88021f47b98488a9da2854033f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40095 } } peers { permanent_uuid: "6f4786a3c2c84cd8889e9f9629af0bec" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 33323 } } peers { permanent_uuid: "c9a39d239311471cad59a156ddca6fd6" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44125 } }
I20260806 00:55:37.542024 11250 raft_consensus.cc:374] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:55:37.542083 11250 raft_consensus.cc:729] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9a39d239311471cad59a156ddca6fd6, State: Initialized, Role: FOLLOWER
I20260806 00:55:37.542567 11250 consensus_queue.cc:260] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d813d88021f47b98488a9da2854033f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40095 } } peers { permanent_uuid: "6f4786a3c2c84cd8889e9f9629af0bec" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 33323 } } peers { permanent_uuid: "c9a39d239311471cad59a156ddca6fd6" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44125 } }
I20260806 00:55:37.543638 11250 ts_tablet_manager.cc:1428] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 00:55:37.543982 11224 heartbeater.cc:503] Master 127.27.77.254:33531 was elected leader, sending a full tablet report...
I20260806 00:55:37.543993 11255 tablet_bootstrap.cc:492] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec: Bootstrap starting.
I20260806 00:55:37.546851 11255 tablet_bootstrap.cc:654] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec: Neither blocks nor log segments found. Creating new log.
I20260806 00:55:37.547940 11255 log.cc:824] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec: Log is configured to *not* fsync() on all Append() calls
I20260806 00:55:37.550097 11255 tablet_bootstrap.cc:492] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec: No bootstrap required, opened a new log
I20260806 00:55:37.550349 11255 ts_tablet_manager.cc:1397] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:55:37.554276 11255 raft_consensus.cc:348] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d813d88021f47b98488a9da2854033f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40095 } } peers { permanent_uuid: "6f4786a3c2c84cd8889e9f9629af0bec" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 33323 } } peers { permanent_uuid: "c9a39d239311471cad59a156ddca6fd6" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44125 } }
I20260806 00:55:37.554550 11255 raft_consensus.cc:374] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:55:37.554620 11255 raft_consensus.cc:729] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4786a3c2c84cd8889e9f9629af0bec, State: Initialized, Role: FOLLOWER
I20260806 00:55:37.555119 11255 consensus_queue.cc:260] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d813d88021f47b98488a9da2854033f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40095 } } peers { permanent_uuid: "6f4786a3c2c84cd8889e9f9629af0bec" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 33323 } } peers { permanent_uuid: "c9a39d239311471cad59a156ddca6fd6" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44125 } }
I20260806 00:55:37.555858 11053 heartbeater.cc:503] Master 127.27.77.254:33531 was elected leader, sending a full tablet report...
I20260806 00:55:37.556257 11255 ts_tablet_manager.cc:1428] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260806 00:55:37.593331 11054 tablet.cc:2369] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 00:55:37 dist-test-slave-sb2z krb5kdc[9993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977736, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
W20260806 00:55:37.645884 11225 tablet.cc:2369] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:55:37.721454 10881 tablet.cc:2369] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:55:37.731709 11256 raft_consensus.cc:482] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:55:37.731976 11256 raft_consensus.cc:504] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d813d88021f47b98488a9da2854033f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40095 } } peers { permanent_uuid: "6f4786a3c2c84cd8889e9f9629af0bec" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 33323 } } peers { permanent_uuid: "c9a39d239311471cad59a156ddca6fd6" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44125 } }
I20260806 00:55:37.733433 11256 leader_election.cc:288] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4786a3c2c84cd8889e9f9629af0bec (127.27.77.194:33323), c9a39d239311471cad59a156ddca6fd6 (127.27.77.195:44125)
I20260806 00:55:37.743207 11243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.734278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:33669 (local address 127.27.77.194:33323)
0806 00:55:37.734434 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 00:55:37.734439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:55:37.734987 (+ 548us) server_negotiation.cc:407] Connection header received
0806 00:55:37.735425 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:55:37.735428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:55:37.735487 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:55:37.735566 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:55:37.736059 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.736822 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.741769 (+ 4947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.742415 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.742525 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:37.742584 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 00:55:37.742952 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 00:55:37.743059 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:55:37.743741 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.733823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:33323 (local address 127.27.77.193:33669)
0806 00:55:37.734886 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0806 00:55:37.734902 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:55:37.735277 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:55:37.735722 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:55:37.735726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:55:37.735746 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:55:37.735914 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:37.735921 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.736999 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.737002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:37.738359 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:37.738370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.742700 (+ 4330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.742703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:37.742801 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:37.742826 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 00:55:37.743454 (+ 628us) client_negotiation.cc:240] Negotiation successful
0806 00:55:37.743562 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"thread_start_us":126,"threads_started":1}
I20260806 00:55:37.744462 10997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b5950effb674f69a18eb9a674411875" candidate_uuid: "4d813d88021f47b98488a9da2854033f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4786a3c2c84cd8889e9f9629af0bec" is_pre_election: true
I20260806 00:55:37.744830 10997 raft_consensus.cc:2454] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d813d88021f47b98488a9da2854033f in term 0.
I20260806 00:55:37.745431 10734 leader_election.cc:302] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d813d88021f47b98488a9da2854033f, 6f4786a3c2c84cd8889e9f9629af0bec; no voters:
I20260806 00:55:37.745782 11256 raft_consensus.cc:2781] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:55:37.745872 11256 raft_consensus.cc:482] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:55:37.745921 11256 raft_consensus.cc:3037] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:55:37.747440 11256 raft_consensus.cc:504] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d813d88021f47b98488a9da2854033f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40095 } } peers { permanent_uuid: "6f4786a3c2c84cd8889e9f9629af0bec" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 33323 } } peers { permanent_uuid: "c9a39d239311471cad59a156ddca6fd6" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44125 } }
I20260806 00:55:37.747869 11256 leader_election.cc:288] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [CANDIDATE]: Term 1 election: Requested vote from peers 6f4786a3c2c84cd8889e9f9629af0bec (127.27.77.194:33323), c9a39d239311471cad59a156ddca6fd6 (127.27.77.195:44125)
I20260806 00:55:37.748378 10997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b5950effb674f69a18eb9a674411875" candidate_uuid: "4d813d88021f47b98488a9da2854033f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4786a3c2c84cd8889e9f9629af0bec"
I20260806 00:55:37.748519 10997 raft_consensus.cc:3037] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:55:37.750022 10997 raft_consensus.cc:2454] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d813d88021f47b98488a9da2854033f in term 1.
I20260806 00:55:37.750439 10734 leader_election.cc:302] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d813d88021f47b98488a9da2854033f, 6f4786a3c2c84cd8889e9f9629af0bec; no voters:
I20260806 00:55:37.750648 11256 raft_consensus.cc:2781] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:55:37.750902 11256 raft_consensus.cc:686] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [term 1 LEADER]: Becoming Leader. State: Replica: 4d813d88021f47b98488a9da2854033f, State: Running, Role: LEADER
I20260806 00:55:37.751325 11256 consensus_queue.cc:237] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d813d88021f47b98488a9da2854033f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40095 } } peers { permanent_uuid: "6f4786a3c2c84cd8889e9f9629af0bec" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 33323 } } peers { permanent_uuid: "c9a39d239311471cad59a156ddca6fd6" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44125 } }
I20260806 00:55:37.755928 10606 catalog_manager.cc:5495] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d813d88021f47b98488a9da2854033f (127.27.77.193). New cstate: current_term: 1 leader_uuid: "4d813d88021f47b98488a9da2854033f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d813d88021f47b98488a9da2854033f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f4786a3c2c84cd8889e9f9629af0bec" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 33323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9a39d239311471cad59a156ddca6fd6" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44125 } health_report { overall_health: UNKNOWN } } }
I20260806 00:55:37.758692 11248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.734657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:35685 (local address 127.27.77.195:44125)
0806 00:55:37.734779 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 00:55:37.734784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:55:37.738696 (+ 3912us) server_negotiation.cc:407] Connection header received
0806 00:55:37.738883 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:55:37.738886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:55:37.738937 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:55:37.738996 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:55:37.739380 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.740185 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.757231 (+ 17046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.757861 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.757987 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:37.758051 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 00:55:37.758465 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 00:55:37.758559 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:55:37.759676 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.734519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:44125 (local address 127.27.77.193:35685)
0806 00:55:37.738606 (+ 4087us) negotiation.cc:107] Waiting for socket to connect
0806 00:55:37.738621 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:55:37.738768 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:55:37.739131 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:55:37.739133 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:55:37.739150 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:55:37.739261 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:37.739266 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.740355 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.740358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:37.741609 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:55:37.741619 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.758179 (+ 16560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.758183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:55:37.758302 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:37.758328 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:55:37.758933 (+ 605us) client_negotiation.cc:240] Negotiation successful
0806 00:55:37.759487 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4012,"thread_start_us":109,"threads_started":1}
I20260806 00:55:37.760388 11168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b5950effb674f69a18eb9a674411875" candidate_uuid: "4d813d88021f47b98488a9da2854033f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9a39d239311471cad59a156ddca6fd6" is_pre_election: true
I20260806 00:55:37.760756 11168 raft_consensus.cc:2454] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d813d88021f47b98488a9da2854033f in term 0.
I20260806 00:55:37.761178 11167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b5950effb674f69a18eb9a674411875" candidate_uuid: "4d813d88021f47b98488a9da2854033f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9a39d239311471cad59a156ddca6fd6"
I20260806 00:55:37.761299 11167 raft_consensus.cc:3037] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:55:37.762732 11167 raft_consensus.cc:2454] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d813d88021f47b98488a9da2854033f in term 1.
I20260806 00:55:37.835809 11241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.830295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40864 (local address 127.27.77.193:40095)
0806 00:55:37.830459 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 00:55:37.830465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:55:37.830484 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:55:37.830584 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:55:37.830588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:55:37.830674 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:55:37.830809 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:55:37.831124 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.832091 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.833473 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:55:37.833827 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:55:37.833923 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:55:37.834381 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:55:37.834979 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:55:37.835112 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 00:55:37.835569 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 00:55:37.835639 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:55:37.842979 11167 raft_consensus.cc:1261] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6 [term 1 FOLLOWER]: Refusing update from remote peer 4d813d88021f47b98488a9da2854033f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:55:37.842979 10997 raft_consensus.cc:1261] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec [term 1 FOLLOWER]: Refusing update from remote peer 4d813d88021f47b98488a9da2854033f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:55:37.843801 11265 consensus_queue.cc:1035] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9a39d239311471cad59a156ddca6fd6" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:55:37.844274 11265 consensus_queue.cc:1035] T 4b5950effb674f69a18eb9a674411875 P 4d813d88021f47b98488a9da2854033f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4786a3c2c84cd8889e9f9629af0bec" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 33323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:55:37.868578 11270 mvcc.cc:205] Tried to move back new op lower bound from 7315364814196772864 to 7315364813833621504. Current Snapshot: MvccSnapshot[applied={T|T < 7315364814196772864}]
I20260806 00:55:37.877089 11271 mvcc.cc:205] Tried to move back new op lower bound from 7315364814196772864 to 7315364813833621504. Current Snapshot: MvccSnapshot[applied={T|T < 7315364814196772864}]
I20260806 00:55:37.877324 11273 mvcc.cc:205] Tried to move back new op lower bound from 7315364814196772864 to 7315364813833621504. Current Snapshot: MvccSnapshot[applied={T|T < 7315364814196772864}]
I20260806 00:55:37.890221 10606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:55:37.890460 10606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:55:37.893504 10606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a0b3aed7f3d4d2d881eb034680b8f25: Sending DeleteTablet for 3 replicas of tablet 4b5950effb674f69a18eb9a674411875
I20260806 00:55:37.894878 11129 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b5950effb674f69a18eb9a674411875 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:55:37 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:60784
I20260806 00:55:37.895020 10970 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b5950effb674f69a18eb9a674411875 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:55:37 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:35336
I20260806 00:55:37.895501 10792 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b5950effb674f69a18eb9a674411875 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:55:37 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:40862
I20260806 00:55:37.896166 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 10709
I20260806 00:55:37.905107 11280 tablet_replica.cc:330] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6: stopping tablet replica
I20260806 00:55:37.908401 11280 raft_consensus.cc:2229] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:55:37.908996 11280 raft_consensus.cc:2258] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:55:37.910741 11280 ts_tablet_manager.cc:1905] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:55:37.914964 11280 ts_tablet_manager.cc:1918] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:55:37.917999 11280 log.cc:1196] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/4b5950effb674f69a18eb9a674411875
I20260806 00:55:37.918558 11280 ts_tablet_manager.cc:1939] T 4b5950effb674f69a18eb9a674411875 P c9a39d239311471cad59a156ddca6fd6: Deleting consensus metadata
W20260806 00:55:37.920653 10592 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:40095 (error 108)
I20260806 00:55:37.921088 11281 tablet_replica.cc:330] T 4b5950effb674f69a18eb9a674411875 P 6f4786a3c2c84cd8889e9f9629af0bec: stopping tablet replica
I20260806 00:55:37.922266 10592 catalog_manager.cc:4841] TS c9a39d239311471cad59a156ddca6fd6 (127.27.77.195:44125): tablet 4b5950effb674f69a18eb9a674411875 (table test-table [id=faac6df70d334869b93da346cdcfa88a]) successfully deleted
I20260806 00:55:37.922438 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.922067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:40095 (local address 127.0.0.1:40878)
0806 00:55:37.922241 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 00:55:37.922338 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:40095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 00:55:37.922632 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 10886
W20260806 00:55:37.922663 10592 catalog_manager.cc:4584] TS 4d813d88021f47b98488a9da2854033f (127.27.77.193:40095): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4b5950effb674f69a18eb9a674411875: Network error: Client connection negotiation failed: client connection to 127.27.77.193:40095: connect: Connection refused (error 111)
I20260806 00:55:37.968500 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 11059
I20260806 00:55:37.969065 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.968598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:33323 (local address 127.0.0.1:35342)
0806 00:55:37.968854 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 00:55:37.968942 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.194:33323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260806 00:55:37.983907 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.978097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:40095 (local address 127.0.0.1:40894)
0806 00:55:37.983710 (+ 5613us) negotiation.cc:107] Waiting for socket to connect
0806 00:55:37.983793 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:40095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5485}
I20260806 00:55:37.985070 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:55:37.984723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:40095 (local address 127.0.0.1:40908)
0806 00:55:37.984913 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 00:55:37.984990 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:40095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260806 00:55:37.986393 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 10571
I20260806 00:55:38.000253 27959 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 00:55:38.357316 27959 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 00:55:38.357517 27959 mini_ranger.cc:66] Stopping Ranger...
I20260806 00:55:38.713028 27959 mini_ranger.cc:68] Stopped Ranger
2026-08-06 00:55:38.713 UTC [10028] LOG: received smart shutdown request
2026-08-06 00:55:38.717 UTC [10028] LOG: background worker "logical replication launcher" (PID 10043) exited with exit code 1
2026-08-06 00:55:38.717 UTC [10035] LOG: shutting down
2026-08-06 00:55:38.927 UTC [10028] LOG: database system is shut down
2026-08-06T00:55:38Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51532 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:55:39 dist-test-slave-sb2z krb5kdc[11294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:55:39 dist-test-slave-sb2z krb5kdc[11294](info): set up 2 sockets
Aug 06 00:55:39 dist-test-slave-sb2z krb5kdc[11294](info): commencing operation
krb5kdc: starting...
W20260806 00:55:40.280316 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:55:40 dist-test-slave-sb2z krb5kdc[11294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:55:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:55:40Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.27.77.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.27.77.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/rangeradmin_127.27.77.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/rangeradmin_127.27.77.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.27.77.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.27.77.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/rangerlookup_127.27.77.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/rangerlookup_127.27.77.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.77.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.27.77.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/HTTP_127.27.77.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/HTTP_127.27.77.212@KRBTEST.COM.keytab.
I20260806 00:55:40.470556 27959 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskxB88Ks/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/postgres -l logfile start
2026-08-06 00:55:43.137 UTC [11330] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 00:55:43.137 UTC [11330] LOG: listening on IPv4 address "127.27.77.212", port 42285
2026-08-06 00:55:43.141 UTC [11330] LOG: listening on Unix socket "/tmp/.s.PGSQL.42285"
2026-08-06 00:55:43.164 UTC [11333] LOG: database system was shut down at 2026-08-06 00:55:41 UTC
2026-08-06 00:55:43.169 UTC [11330] LOG: database system is ready to accept connections
I20260806 00:55:44.309630 27959 mini_postgres.cc:96] Postgres bound to 42285
2026-08-06 00:55:44.317 UTC [11341] FATAL: database "slave" does not exist
127.27.77.212:42285 - accepting connections
I20260806 00:55:44.318342 27959 mini_ranger.cc:161] Starting Ranger...
I20260806 00:55:44.335711 27959 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 00:55:44.869411 27959 mini_ranger.cc:87] Created ranger Postgres database
I20260806 00:55:44.869513 27959 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-admin
2026-08-06 00:55:45,167 [I] DB FLAVOR :POSTGRES
2026-08-06 00:55:45,167 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 00:55:45,168 [I] Checking connection..
2026-08-06 00:55:45,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 00:55:45,444 [I] Checking connection passed.
2026-08-06 00:55:45,445 [I] --------- Verifying version history table ---------
2026-08-06 00:55:45,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:55:45,741 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 00:55:45,742 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:55:46,025 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 00:55:46,025 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 00:55:46,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 00:55:46.277 UTC [11429] WARNING: there is no transaction in progress
2026-08-06 00:55:46,286 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 00:55:46,287 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:55:46,560 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 00:55:46,560 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 00:55:46,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 00:55:46,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 00:55:47,072 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-sb2z.c.gcp-upstream.internal','N') ;"
2026-08-06 00:55:47,335 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 00:55:47,619 [I] Table x_portal_user does not exist in database ranger
2026-08-06 00:55:47,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 00:55:47,881 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 00:55:47,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 00:55:48,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 00:55:48,407 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 00:55:48,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 00:55:49.438 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:49.455 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:49.473 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:49.489 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:49.505 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:49.524 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:49.581 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:49.588 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:49.603 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:49.613 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:49.624 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:49.637 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:49.647 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:49.653 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:49.659 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:49.666 UTC [11565] WARNING: there is no transaction in progress
2026-08-06 00:55:50,378 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 00:55:50,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 00:55:50,652 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 00:55:50,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 00:55:50,931 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 00:55:50,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 00:55:51,189 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 00:55:51,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 00:55:51,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 00:55:51,739 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-sb2z.c.gcp-upstream.internal';"
2026-08-06 00:55:51,994 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 00:55:51,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 00:55:52,267 [I] DB_PATCHES have already been applied
I20260806 00:55:52.273967 27959 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-admin:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskxB88Ks/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 00:55:52.274047 27959 mini_ranger.cc:193] Using host: 127.27.77.212
I20260806 00:55:52.278725 27959 mini_ranger.cc:227] Ranger admin URL: http://127.27.77.212:39429
Aug 06, 2026 12:55:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:55:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:55:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 12:55:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 12:55:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 12:55:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 12:55:53 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 00:55:53.618175 27959 mini_ranger.cc:160] Time spent starting Ranger: real 9.300s user 0.000s sys 0.014s
Aug 06, 2026 12:56:15 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 12:56:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 12:56:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 12:56:15 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 12:56:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 12:56:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 12:56:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 12:56:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 12:56:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 12:56:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 12:56:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 12:56:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 12:56:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 12:56:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 12:56:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 12:56:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 12:56:15 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 12:56:16 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 00:56:17.065480 27959 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.27.77.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.27.77.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/rangerkms_127.27.77.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.27.77.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/rangerkms_127.27.77.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.77.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.27.77.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.77.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/HTTP_127.27.77.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.77.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/HTTP_127.27.77.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 00:56:17 dist-test-slave-sb2z krb5kdc[11294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977777, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 00:56:17.157864 27959 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 00:56:17.168949 27959 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 00:56:18.086433 27959 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 00:56:18.086527 27959 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-kms
I20260806 00:56:18.086551 27959 mini_ranger_kms.cc:227] Using postgres at 127.27.77.212:42285
2026-08-06 00:56:18,288 [I] DB FLAVOR :POSTGRES
2026-08-06 00:56:18,289 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 00:56:18,289 [I] Checking connection
2026-08-06 00:56:18,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 00:56:18,557 [I] connection success
2026-08-06 00:56:18,558 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 00:56:18,558 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 00:56:18,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 00:56:18,829 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 00:56:18,829 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 00:56:18,829 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 00:56:18,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.77.212:42285/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 00:56:19,154 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 00:56:19.699175 27959 mini_ranger_kms.cc:310] Created kms service
I20260806 00:56:19.916227 27959 mini_ranger_kms.cc:326] Created kudu user
I20260806 00:56:19.965133 27959 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 00:56:19 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977777, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:56:20.384594 27959 mini_ranger_kms.cc:384] Added ranger policy
I20260806 00:56:20.384764 27959 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-kms:/tmp/dist-test-taskxB88Ks/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskxB88Ks/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 00:56:20.384790 27959 mini_ranger_kms.cc:240] Using host: 127.27.77.212
I20260806 00:56:20.390046 27959 mini_ranger_kms.cc:276] Ranger KMS PID: 11808
I20260806 00:56:20.390148 27959 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.27.77.212:47485
Aug 06, 2026 12:56:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:56:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:56:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 12:56:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 12:56:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 12:56:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 00:56:22 dist-test-slave-sb2z krb5kdc[11294](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785977782, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.77.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:23.031672 27959 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.874s user 0.009s sys 0.179s
I20260806 00:56:23.031832 27959 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 00:56:23.031919 27959 mini_ranger_kms.cc:401] 127.27.77.212:47485/kms/v1/keys
Aug 06 00:56:23 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977782, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.77.212@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
Aug 06 00:56:23 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977782, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.77.212@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
Aug 06 00:56:25 dist-test-slave-sb2z krb5kdc[11294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:56:26.022540 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:45213
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:35045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:45213
--ranger_config_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.77.212:47485/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:26.184161 11874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:26.184589 11874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:26.184695 11874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:26.188985 11874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:56:26.189073 11874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:26.189134 11874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:26.189172 11874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:56:26.189211 11874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:56:26.195163 11874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:35045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/wal
--ranger_kms_url=127.27.77.212:47485/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:45213
--ranger_config_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:45213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:26.196342 11874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:26.197430 11874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:56:26.207723 11874 server_base.cc:1034] running on GCE node
W20260806 00:56:26.207754 11879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:26.207808 11880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:26.208105 11882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:26.208768 11874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:26.209942 11874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:26.211182 11874 hybrid_clock.cc:648] HybridClock initialized: now 1785977786211134 us; error 37 us; skew 500 ppm
Aug 06 00:56:26 dist-test-slave-sb2z krb5kdc[11294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977786, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:26.214457 11874 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:56:26.215708 11874 webserver.cc:460] Webserver started at http://127.27.77.254:33473/ using document root <none> and password file <none>
I20260806 00:56:26.216429 11874 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:26.216514 11874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:26.216745 11874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:56:26 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977786, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:56:26.276017 11874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "cb3cfca399c14b1d95780b5fdff28e83"
format_stamp: "Formatted at 2026-08-06 00:56:26 on dist-test-slave-sb2z"
server_key: "8a8fb7b95bf98def996f76797cf57044"
server_key_iv: "3726f908fc2ee2210348199b7f04e963"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:26.276710 11874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "cb3cfca399c14b1d95780b5fdff28e83"
format_stamp: "Formatted at 2026-08-06 00:56:26 on dist-test-slave-sb2z"
server_key: "8a8fb7b95bf98def996f76797cf57044"
server_key_iv: "3726f908fc2ee2210348199b7f04e963"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:26.281708 11874 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 00:56:26.296729 11889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:26.298151 11874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:56:26.298308 11874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "cb3cfca399c14b1d95780b5fdff28e83"
format_stamp: "Formatted at 2026-08-06 00:56:26 on dist-test-slave-sb2z"
server_key: "8a8fb7b95bf98def996f76797cf57044"
server_key_iv: "3726f908fc2ee2210348199b7f04e963"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:26.298451 11874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:26.324844 11874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:26.328069 11874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:26.328285 11874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:26.336762 11874 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:45213
I20260806 00:56:26.336769 11958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:45213 every 8 connection(s)
I20260806 00:56:26.337778 11874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:56:26.340678 11960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:26.343775 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 11874
I20260806 00:56:26.343911 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:56:26.344237 27959 external_mini_cluster.cc:1428] Setting key a0a59d9371d3a7c5b3455c5356df5a6e
I20260806 00:56:26.347035 11960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83: Bootstrap starting.
I20260806 00:56:26.349723 11960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:26.350487 11960 log.cc:824] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:26.352691 11960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83: No bootstrap required, opened a new log
Aug 06 00:56:26 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:26.357303 11960 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cfca399c14b1d95780b5fdff28e83" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45213 } }
I20260806 00:56:26.357594 11960 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:26.357718 11960 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb3cfca399c14b1d95780b5fdff28e83, State: Initialized, Role: FOLLOWER
I20260806 00:56:26.358268 11960 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cfca399c14b1d95780b5fdff28e83" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45213 } }
I20260806 00:56:26.358460 11960 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:56:26.358580 11960 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:56:26.358695 11960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:26.360060 11960 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cfca399c14b1d95780b5fdff28e83" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45213 } }
I20260806 00:56:26.360450 11960 leader_election.cc:302] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb3cfca399c14b1d95780b5fdff28e83; no voters:
I20260806 00:56:26.360790 11960 leader_election.cc:288] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:56:26.360951 11966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:56:26.361232 11966 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 1 LEADER]: Becoming Leader. State: Replica: cb3cfca399c14b1d95780b5fdff28e83, State: Running, Role: LEADER
I20260806 00:56:26.361581 11966 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cfca399c14b1d95780b5fdff28e83" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45213 } }
I20260806 00:56:26.361891 11964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:26.345753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37446 (local address 127.27.77.254:45213)
0806 00:56:26.346296 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:26.346315 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:26.346360 (+ 45us) server_negotiation.cc:407] Connection header received
0806 00:56:26.347191 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:26.347215 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:26.347688 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:26.348059 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:26.349014 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:26.350616 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:26.351743 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:26.352506 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:26.352670 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:26.355828 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:26.355835 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:26.355857 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:26.355892 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:26.357947 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:26.358491 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:26.358498 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:26.358505 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:26.358648 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:26.358987 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:26.358991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:26.358992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:26.359511 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:26.359779 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:26.360579 (+ 800us) server_negotiation.cc:299] Negotiation successful
0806 00:56:26.360877 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":153,"threads_started":1}
I20260806 00:56:26.362257 11960 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:56:26.362802 11966 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb3cfca399c14b1d95780b5fdff28e83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cfca399c14b1d95780b5fdff28e83" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45213 } } }
I20260806 00:56:26.362985 11966 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:26.363238 11966 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb3cfca399c14b1d95780b5fdff28e83. Latest consensus state: current_term: 1 leader_uuid: "cb3cfca399c14b1d95780b5fdff28e83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cfca399c14b1d95780b5fdff28e83" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45213 } } }
I20260806 00:56:26.364486 11966 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:26.364658 11960 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 00:56:26.364840 11972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:56:26.373826 11972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:56:26.382175 11972 catalog_manager.cc:1347] Generated new cluster ID: 35e5011e198b455db3cd9949fcaf4a8c
I20260806 00:56:26.382247 11972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:56:26.416958 11972 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:56:26.418401 11972 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:56:26.423542 11972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83: Generated new TSK 0
I20260806 00:56:26.424429 11972 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:56:27.183640 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.77.212:47485/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:45213
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:35045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:27.309829 12019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:27.310179 12019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:27.310254 12019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:27.314766 12019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:27.314854 12019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:27.314960 12019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:56:27.323753 12019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:35045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.27.77.212:47485/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:45213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:27.325064 12019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:27.326128 12019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:27.336344 12029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:27.336344 12025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:27.336705 12026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:27.337536 12019 server_base.cc:1034] running on GCE node
I20260806 00:56:27.337996 12019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:27.338819 12019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:27.342792 12019 hybrid_clock.cc:648] HybridClock initialized: now 1785977787342747 us; error 49 us; skew 500 ppm
Aug 06 00:56:27 dist-test-slave-sb2z krb5kdc[11294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977787, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:27.346462 12019 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:56:27.347723 12019 webserver.cc:460] Webserver started at http://127.27.77.193:38153/ using document root <none> and password file <none>
I20260806 00:56:27.348538 12019 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:27.348696 12019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:27.348997 12019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:56:27 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977787, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:56:27.364701 12019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "45041559aa2648eeac724b33ae61b165"
format_stamp: "Formatted at 2026-08-06 00:56:27 on dist-test-slave-sb2z"
server_key: "62fb3b3a4cde523f4b8519c4aab67288"
server_key_iv: "2b4f09cb7933a74719ec2f858f53b1b0"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:27.365559 12019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "45041559aa2648eeac724b33ae61b165"
format_stamp: "Formatted at 2026-08-06 00:56:27 on dist-test-slave-sb2z"
server_key: "62fb3b3a4cde523f4b8519c4aab67288"
server_key_iv: "2b4f09cb7933a74719ec2f858f53b1b0"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:27.375453 12019 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.004s
I20260806 00:56:27.392709 12038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:27.394560 12019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:56:27.394696 12019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "45041559aa2648eeac724b33ae61b165"
format_stamp: "Formatted at 2026-08-06 00:56:27 on dist-test-slave-sb2z"
server_key: "62fb3b3a4cde523f4b8519c4aab67288"
server_key_iv: "2b4f09cb7933a74719ec2f858f53b1b0"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:27.394831 12019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:27.459261 12019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:27.465641 12019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:27.466041 12019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:27.466913 12019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:27.468106 12019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:27.468214 12019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:27.468350 12019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:27.468453 12019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:27.481894 12019 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:45415
I20260806 00:56:27.483268 12019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:56:27.484632 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 12019
I20260806 00:56:27.484768 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:56:27.485102 27959 external_mini_cluster.cc:1428] Setting key 48d1111066f4781561af33ee809c58a2
I20260806 00:56:27.485430 12182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:45415 every 8 connection(s)
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
Aug 06 00:56:27 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977787, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:27.520831 12188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:27.492863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:42529 (local address 127.27.77.254:45213)
0806 00:56:27.493140 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:27.493145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:27.502109 (+ 8964us) server_negotiation.cc:407] Connection header received
0806 00:56:27.507795 (+ 5686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:27.507800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:27.507878 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:27.507970 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:27.509831 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:27.510743 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:27.512561 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:27.512942 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:27.513028 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:27.516248 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:27.516252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:27.516255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:27.516285 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:27.518254 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:27.518933 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:27.518937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:27.518939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:27.519034 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:27.519457 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:27.519461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:27.519463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:27.519710 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:27.519794 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:27.520495 (+ 701us) server_negotiation.cc:299] Negotiation successful
0806 00:56:27.520635 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":94,"threads_started":1}
I20260806 00:56:27.523180 12187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:27.491657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:45213 (local address 127.27.77.193:42529)
0806 00:56:27.501860 (+ 10203us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:27.501965 (+ 105us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:27.503347 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:27.508483 (+ 5136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:27.508494 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:27.508899 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:27.509635 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:27.509655 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:27.510934 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:27.510937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:27.512402 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:27.512414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:27.513265 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:27.513269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:27.513477 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:27.514167 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:27.514190 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:27.516033 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:27.518406 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:27.518412 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:27.518430 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:27.518794 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:27.519153 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:27.519155 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:27.519157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:27.519339 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:27.519864 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:27.519870 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:27.520199 (+ 329us) client_negotiation.cc:769] Sending connection context
0806 00:56:27.521347 (+ 1148us) client_negotiation.cc:240] Negotiation successful
0806 00:56:27.521711 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9121,"thread_start_us":147,"threads_started":1}
I20260806 00:56:27.524946 12183 heartbeater.cc:347] Connected to a master server at 127.27.77.254:45213
I20260806 00:56:27.525204 12183 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:27.525785 12183 heartbeater.cc:511] Master 127.27.77.254:45213 requested a full tablet report, sending...
I20260806 00:56:27.527952 11910 ts_manager.cc:194] Registered new tserver with Master: 45041559aa2648eeac724b33ae61b165 (127.27.77.193:45415)
I20260806 00:56:27.529533 11910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:42529
Aug 06 00:56:27 dist-test-slave-sb2z krb5kdc[11294](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785977787, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:56:27.554937 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.77.212:47485/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:45213
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:35045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:27.718041 12190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:27.718834 12190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:27.718974 12190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:27.724920 12190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:27.725116 12190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:27.725245 12190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:56:27.732653 12190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:35045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.27.77.212:47485/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:45213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:27.734306 12190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:27.735867 12190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:27.749030 12197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:27.749553 12190 server_base.cc:1034] running on GCE node
W20260806 00:56:27.749948 12196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:27.750581 12199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:27.751114 12190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:27.751945 12190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:27.753122 12190 hybrid_clock.cc:648] HybridClock initialized: now 1785977787753051 us; error 89 us; skew 500 ppm
Aug 06 00:56:27 dist-test-slave-sb2z krb5kdc[11294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977787, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:27.756867 12190 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:56:27.757948 12190 webserver.cc:460] Webserver started at http://127.27.77.194:37597/ using document root <none> and password file <none>
I20260806 00:56:27.758656 12190 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:27.758729 12190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:27.758941 12190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:56:27 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977787, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:56:27.777552 12190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "00d0abd0377648f98771bb0d9b281677"
format_stamp: "Formatted at 2026-08-06 00:56:27 on dist-test-slave-sb2z"
server_key: "056bbab0b2995fcda41f0a9429e0a511"
server_key_iv: "16f797eebc21e2440b0c2a0b1bd8cef8"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:27.779954 12190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "00d0abd0377648f98771bb0d9b281677"
format_stamp: "Formatted at 2026-08-06 00:56:27 on dist-test-slave-sb2z"
server_key: "056bbab0b2995fcda41f0a9429e0a511"
server_key_iv: "16f797eebc21e2440b0c2a0b1bd8cef8"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:27.785753 12190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 00:56:27.807932 12208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:27.814924 12190 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.003s sys 0.001s
I20260806 00:56:27.815191 12190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "00d0abd0377648f98771bb0d9b281677"
format_stamp: "Formatted at 2026-08-06 00:56:27 on dist-test-slave-sb2z"
server_key: "056bbab0b2995fcda41f0a9429e0a511"
server_key_iv: "16f797eebc21e2440b0c2a0b1bd8cef8"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:27.815419 12190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:27.857045 12190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:27.860415 12190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:27.860649 12190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:27.861380 12190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:27.862416 12190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:27.862534 12190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 00:56:27.862659 12190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:27.862725 12190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:27.874459 12190 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:35989
I20260806 00:56:27.875590 12190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:56:27.879236 12354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:35989 every 8 connection(s)
I20260806 00:56:27.885859 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 12190
I20260806 00:56:27.886014 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:56:27.886328 27959 external_mini_cluster.cc:1428] Setting key 2f41909a98b375e78e3520be03ca8f3b
Aug 06 00:56:27 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977787, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:27.912735 12188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:27.877604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:53023 (local address 127.27.77.254:45213)
0806 00:56:27.877754 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:27.877759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:27.882732 (+ 4973us) server_negotiation.cc:407] Connection header received
0806 00:56:27.884197 (+ 1465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:27.884201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:27.884261 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:27.884334 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:27.899347 (+ 15013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:27.900266 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:27.902083 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:27.902430 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:27.902518 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:27.908376 (+ 5858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:27.908380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:27.908383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:27.908413 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:27.910170 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:27.910824 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:27.910828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:27.910830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:27.910920 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:27.911339 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:27.911343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:27.911344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:27.911627 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:27.911715 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:27.912407 (+ 692us) server_negotiation.cc:299] Negotiation successful
0806 00:56:27.912561 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:56:27.914678 12358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:27.878102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:45213 (local address 127.27.77.194:53023)
0806 00:56:27.878624 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:27.878682 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:27.883801 (+ 5119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:27.897433 (+ 13632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:27.897457 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:27.897875 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:27.898620 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:27.898640 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:27.900418 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:27.900422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:27.901928 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:27.901941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:27.902746 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:27.902750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:27.902951 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:27.906350 (+ 3399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:27.906373 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:27.908156 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:27.910324 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:27.910331 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:27.910349 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:27.910702 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:27.911027 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:27.911030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:27.911033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:27.911224 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:27.911783 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:27.911790 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:27.912123 (+ 333us) client_negotiation.cc:769] Sending connection context
0806 00:56:27.913239 (+ 1116us) client_negotiation.cc:240] Negotiation successful
0806 00:56:27.913610 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":149,"threads_started":1}
I20260806 00:56:27.916311 12355 heartbeater.cc:347] Connected to a master server at 127.27.77.254:45213
I20260806 00:56:27.916574 12355 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:27.917130 12355 heartbeater.cc:511] Master 127.27.77.254:45213 requested a full tablet report, sending...
I20260806 00:56:27.918334 11910 ts_manager.cc:194] Registered new tserver with Master: 00d0abd0377648f98771bb0d9b281677 (127.27.77.194:35989)
I20260806 00:56:27.919049 11910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:53023
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:56:27.955776 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.77.212:47485/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:45213
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:35045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:56:28 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977787, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
W20260806 00:56:28.136742 12361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:28.137306 12361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:28.137568 12361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:28.144529 12361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:28.144622 12361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:28.144721 12361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:56:28.153497 12361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:35045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.27.77.212:47485/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:45213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:28.155272 12361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:28.156829 12361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:28.170799 12369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:28.171348 12367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:28.170819 12371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:28.172973 12361 server_base.cc:1034] running on GCE node
I20260806 00:56:28.173481 12361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:28.174242 12361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:28.175439 12361 hybrid_clock.cc:648] HybridClock initialized: now 1785977788175406 us; error 50 us; skew 500 ppm
Aug 06 00:56:28 dist-test-slave-sb2z krb5kdc[11294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977788, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:28.178791 12361 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:56:28.179960 12361 webserver.cc:460] Webserver started at http://127.27.77.195:40529/ using document root <none> and password file <none>
I20260806 00:56:28.180614 12361 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:28.180724 12361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:28.181015 12361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:56:28 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977788, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:56:28.198721 12361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "468ac47b21ee434a84e9c56e407b4652"
format_stamp: "Formatted at 2026-08-06 00:56:28 on dist-test-slave-sb2z"
server_key: "6d0f57acd8423229d4f91d67dce38e42"
server_key_iv: "cdba438a893e221b0e2d48b0ab659464"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:28.199584 12361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "468ac47b21ee434a84e9c56e407b4652"
format_stamp: "Formatted at 2026-08-06 00:56:28 on dist-test-slave-sb2z"
server_key: "6d0f57acd8423229d4f91d67dce38e42"
server_key_iv: "cdba438a893e221b0e2d48b0ab659464"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:28.205071 12361 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 00:56:28.230752 12380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:28.232654 12361 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.000s
I20260806 00:56:28.232873 12361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "468ac47b21ee434a84e9c56e407b4652"
format_stamp: "Formatted at 2026-08-06 00:56:28 on dist-test-slave-sb2z"
server_key: "6d0f57acd8423229d4f91d67dce38e42"
server_key_iv: "cdba438a893e221b0e2d48b0ab659464"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:28.233109 12361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:28.266618 12361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:28.270471 12361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:28.270712 12361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:28.271457 12361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:28.272684 12361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:28.272758 12361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:28.272817 12361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:28.272842 12361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:28.284236 12361 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:41199
I20260806 00:56:28.284260 12531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:41199 every 8 connection(s)
I20260806 00:56:28.285404 12361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:56:28.291501 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 12361
I20260806 00:56:28.291710 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:56:28.292001 27959 external_mini_cluster.cc:1428] Setting key 47257d86f2681803fed3374df6c9a468
Aug 06 00:56:28 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977788, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:28.302371 12188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:28.287982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:35725 (local address 127.27.77.254:45213)
0806 00:56:28.288151 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:28.288156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:28.289123 (+ 967us) server_negotiation.cc:407] Connection header received
0806 00:56:28.290086 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:28.290089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:28.290150 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:28.290231 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:28.291387 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:28.292333 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:28.293933 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:28.294341 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:28.294454 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:28.297712 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:28.297716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:28.297719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:28.297748 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:28.299612 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:28.300313 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:28.300317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:28.300319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:28.300451 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:28.300885 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:28.300889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:28.300891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:28.301166 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:28.301282 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:28.301997 (+ 715us) server_negotiation.cc:299] Negotiation successful
0806 00:56:28.302153 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:56:28.303627 12534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:28.288397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:45213 (local address 127.27.77.195:35725)
0806 00:56:28.288969 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:28.289013 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:28.289872 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:28.290405 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:28.290412 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:28.290697 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:28.291170 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:28.291183 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:28.292489 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:28.292495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:28.293799 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:28.293810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:28.294671 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:28.294676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:28.294937 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:28.295781 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:28.295806 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:28.297542 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:28.299736 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:28.299742 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:28.299760 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:28.300111 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:28.300565 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:28.300568 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:28.300570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:28.300761 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:28.301361 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:28.301368 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:28.301712 (+ 344us) client_negotiation.cc:769] Sending connection context
0806 00:56:28.301923 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 00:56:28.302261 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":150,"threads_started":1}
I20260806 00:56:28.305099 12532 heartbeater.cc:347] Connected to a master server at 127.27.77.254:45213
I20260806 00:56:28.305430 12532 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:28.306295 12532 heartbeater.cc:511] Master 127.27.77.254:45213 requested a full tablet report, sending...
I20260806 00:56:28.307499 11910 ts_manager.cc:194] Registered new tserver with Master: 468ac47b21ee434a84e9c56e407b4652 (127.27.77.195:41199)
I20260806 00:56:28.308260 11910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:35725
I20260806 00:56:28.318238 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:56:28.319089 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 12019
I20260806 00:56:28.331050 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 12190
I20260806 00:56:28.343343 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 12361
I20260806 00:56:28.357477 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 11874
I20260806 00:56:28.371857 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:45213
--webserver_interface=127.27.77.254
--webserver_port=33473
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:35045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:45213
--ranger_config_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.77.212:47485/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.77.212:47485/kms with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:28.486687 12535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:28.487003 12535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:28.487044 12535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:28.491384 12535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:56:28.491462 12535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:28.491482 12535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:28.491498 12535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:56:28.491514 12535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:56:28.497473 12535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:35045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.77.212:47485/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:45213
--ranger_config_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:45213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=33473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:28.498764 12535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:28.499899 12535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:28.509253 12548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:28.509213 12542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:28.509262 12543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:28.509375 12535 server_base.cc:1034] running on GCE node
I20260806 00:56:28.510177 12535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:28.511348 12535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:28.512581 12535 hybrid_clock.cc:648] HybridClock initialized: now 1785977788512568 us; error 37 us; skew 500 ppm
Aug 06 00:56:28 dist-test-slave-sb2z krb5kdc[11294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977788, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:28.515626 12535 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:56:28.516566 12535 webserver.cc:460] Webserver started at http://127.27.77.254:33473/ using document root <none> and password file <none>
I20260806 00:56:28.517134 12535 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:28.517210 12535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 00:56:28 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977788, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:56:28.561569 12535 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:56:28.564635 12559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:28.566375 12535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:56:28.566541 12535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "cb3cfca399c14b1d95780b5fdff28e83"
format_stamp: "Formatted at 2026-08-06 00:56:26 on dist-test-slave-sb2z"
server_key: "8a8fb7b95bf98def996f76797cf57044"
server_key_iv: "3726f908fc2ee2210348199b7f04e963"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:28.567330 12535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:28.578361 12535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:28.581702 12535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:28.581871 12535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:28.590337 12535 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:45213
I20260806 00:56:28.590435 12627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:45213 every 8 connection(s)
I20260806 00:56:28.591639 12535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:56:28.596119 12628 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:56:28.597905 12628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83: Bootstrap starting.
I20260806 00:56:28.600452 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 12535
I20260806 00:56:28.601006 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.77.212:47485/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:45415
--local_ip_for_outbound_sockets=127.27.77.193
--tserver_master_addrs=127.27.77.254:45213
--webserver_port=38153
--webserver_interface=127.27.77.193
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:35045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.77.212:47485/kms with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:56:28.615839 12628 log.cc:824] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:28.622804 12628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:56:28.623301 12628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83: Bootstrap complete.
I20260806 00:56:28.627938 12628 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cfca399c14b1d95780b5fdff28e83" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45213 } }
I20260806 00:56:28.628355 12628 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb3cfca399c14b1d95780b5fdff28e83, State: Initialized, Role: FOLLOWER
I20260806 00:56:28.628942 12628 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cfca399c14b1d95780b5fdff28e83" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45213 } }
I20260806 00:56:28.629068 12628 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:56:28.629160 12628 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:56:28.629267 12628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:56:28.631250 12628 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cfca399c14b1d95780b5fdff28e83" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45213 } }
I20260806 00:56:28.631678 12628 leader_election.cc:302] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb3cfca399c14b1d95780b5fdff28e83; no voters:
I20260806 00:56:28.632046 12628 leader_election.cc:288] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 00:56:28.632149 12634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 00:56:28.632380 12634 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [term 2 LEADER]: Becoming Leader. State: Replica: cb3cfca399c14b1d95780b5fdff28e83, State: Running, Role: LEADER
I20260806 00:56:28.632754 12634 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cfca399c14b1d95780b5fdff28e83" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45213 } }
I20260806 00:56:28.633389 12628 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:56:28.634276 12635 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cb3cfca399c14b1d95780b5fdff28e83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cfca399c14b1d95780b5fdff28e83" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45213 } } }
I20260806 00:56:28.634413 12635 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:28.635425 12648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:56:28.635552 12637 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb3cfca399c14b1d95780b5fdff28e83. Latest consensus state: current_term: 2 leader_uuid: "cb3cfca399c14b1d95780b5fdff28e83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb3cfca399c14b1d95780b5fdff28e83" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45213 } } }
I20260806 00:56:28.635706 12637 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83 [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:28.636000 12628 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 00:56:28.642525 12648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:56:28.643514 12648 catalog_manager.cc:1259] Loaded cluster ID: 35e5011e198b455db3cd9949fcaf4a8c
I20260806 00:56:28.643669 12648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:56:28.646402 12648 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:56:28.647189 12648 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83: Loaded TSK: 0
I20260806 00:56:28.648559 12648 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:56:28.747625 12632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:28.748080 12632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:28.748232 12632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:28.753881 12632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:28.754041 12632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:28.754205 12632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:56:28.760383 12632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:35045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.77.212:47485/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:45415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=38153
--tserver_master_addrs=127.27.77.254:45213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:28.761556 12632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:28.762634 12632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:28.773562 12685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:28.773502 12682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:28.773880 12683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:28.774212 12632 server_base.cc:1034] running on GCE node
I20260806 00:56:28.774670 12632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:28.775398 12632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:28.776578 12632 hybrid_clock.cc:648] HybridClock initialized: now 1785977788776550 us; error 45 us; skew 500 ppm
Aug 06 00:56:28 dist-test-slave-sb2z krb5kdc[11294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977788, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:28.779665 12632 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:56:28.780833 12632 webserver.cc:460] Webserver started at http://127.27.77.193:38153/ using document root <none> and password file <none>
I20260806 00:56:28.781677 12632 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:28.781832 12632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 00:56:28 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977788, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:56:28.809216 12632 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.000s
I20260806 00:56:28.816741 12694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:28.818723 12632 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.000s
I20260806 00:56:28.818868 12632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "45041559aa2648eeac724b33ae61b165"
format_stamp: "Formatted at 2026-08-06 00:56:27 on dist-test-slave-sb2z"
server_key: "62fb3b3a4cde523f4b8519c4aab67288"
server_key_iv: "2b4f09cb7933a74719ec2f858f53b1b0"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:28.819736 12632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:28.839768 12632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:28.843832 12632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:28.844084 12632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:28.844851 12632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:28.845888 12632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:28.845968 12632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:28.846073 12632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:28.846114 12632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:28.858675 12632 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:45415
I20260806 00:56:28.858774 12840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:45415 every 8 connection(s)
I20260806 00:56:28.860143 12632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:56:28.861558 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 12632
I20260806 00:56:28.862121 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.77.212:47485/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:35989
--local_ip_for_outbound_sockets=127.27.77.194
--tserver_master_addrs=127.27.77.254:45213
--webserver_port=37597
--webserver_interface=127.27.77.194
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:35045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.77.212:47485/kms with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:56:28 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977788, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:28.890511 12843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:28.863104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:45213 (local address 127.27.77.193:37809)
0806 00:56:28.867971 (+ 4867us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:28.868040 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:28.869209 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:28.873567 (+ 4358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:28.873581 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:28.873994 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:28.874782 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:28.874807 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:28.877588 (+ 2781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:28.877596 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:28.879371 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:28.879389 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:28.880797 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:28.880805 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:28.881123 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:28.881938 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:28.881966 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:28.883999 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:28.886677 (+ 2678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:28.886684 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:28.886701 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:28.887035 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:28.887465 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:28.887468 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:28.887470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:28.887684 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:28.888485 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:28.888492 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:28.888808 (+ 316us) client_negotiation.cc:769] Sending connection context
0806 00:56:28.889010 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 00:56:28.889391 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4600,"thread_start_us":164,"threads_started":1}
I20260806 00:56:28.892302 12845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:28.865039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:37809 (local address 127.27.77.254:45213)
0806 00:56:28.872012 (+ 6973us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:28.872037 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:28.872093 (+ 56us) server_negotiation.cc:407] Connection header received
0806 00:56:28.872437 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:28.872456 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:28.873008 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:28.875069 (+ 2061us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:28.875433 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:28.877373 (+ 1940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:28.879607 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:28.880542 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:28.881652 (+ 1110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:28.884280 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:28.884287 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:28.884306 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:28.884340 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:28.886523 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:28.887200 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:28.887206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:28.887211 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:28.887349 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:28.887807 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:28.887811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:28.887813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:28.888341 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:28.890816 (+ 2475us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:28.891011 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 00:56:28.891345 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6774,"thread_start_us":123,"threads_started":1}
I20260806 00:56:28.895090 12841 heartbeater.cc:347] Connected to a master server at 127.27.77.254:45213
I20260806 00:56:28.895364 12841 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:28.896101 12841 heartbeater.cc:511] Master 127.27.77.254:45213 requested a full tablet report, sending...
W20260806 00:56:28.898483 12841 heartbeater.cc:644] Failed to heartbeat to 127.27.77.254:45213 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 00:56:28.898599 12841 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:28.898770 12841 heartbeater.cc:511] Master 127.27.77.254:45213 requested a full tablet report, sending...
I20260806 00:56:28.899801 12841 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:28.899971 12841 heartbeater.cc:511] Master 127.27.77.254:45213 requested a full tablet report, sending...
W20260806 00:56:28.900812 12841 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:56:29.030746 12844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:29.031013 12844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:29.031057 12844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:29.036186 12844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:29.036262 12844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:29.036322 12844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:56:29.042166 12844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:35045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.77.212:47485/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:35989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=37597
--tserver_master_addrs=127.27.77.254:45213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:29.043330 12844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:29.044497 12844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:29.056118 12851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:29.056185 12853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:29.056228 12850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:29.056720 12844 server_base.cc:1034] running on GCE node
I20260806 00:56:29.057255 12844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:29.058157 12844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:29.059432 12844 hybrid_clock.cc:648] HybridClock initialized: now 1785977789059395 us; error 48 us; skew 500 ppm
Aug 06 00:56:29 dist-test-slave-sb2z krb5kdc[11294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977789, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:29.063349 12844 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:56:29.064568 12844 webserver.cc:460] Webserver started at http://127.27.77.194:37597/ using document root <none> and password file <none>
I20260806 00:56:29.065392 12844 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:29.065510 12844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 00:56:29 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977789, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:56:29.096428 12844 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 00:56:29.099900 12865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:29.101756 12844 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:56:29.101922 12844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "00d0abd0377648f98771bb0d9b281677"
format_stamp: "Formatted at 2026-08-06 00:56:27 on dist-test-slave-sb2z"
server_key: "056bbab0b2995fcda41f0a9429e0a511"
server_key_iv: "16f797eebc21e2440b0c2a0b1bd8cef8"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:29.102823 12844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:29.134922 12844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:29.139271 12844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:29.139549 12844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:29.140404 12844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:29.141665 12844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:29.141742 12844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:29.141811 12844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:29.141855 12844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:29.155107 12844 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:35989
I20260806 00:56:29.156920 12844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:56:29.157783 13010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:35989 every 8 connection(s)
I20260806 00:56:29.162374 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 12844
I20260806 00:56:29.162941 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.77.212:47485/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:41199
--local_ip_for_outbound_sockets=127.27.77.195
--tserver_master_addrs=127.27.77.254:45213
--webserver_port=40529
--webserver_interface=127.27.77.195
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:35045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.77.212:47485/kms with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:56:29 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977789, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:29.177353 12845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:29.161716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:38611 (local address 127.27.77.254:45213)
0806 00:56:29.161881 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:29.161887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:29.161907 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:56:29.163058 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:29.163065 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:29.163142 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:29.163218 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:29.165184 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:29.166082 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:29.168083 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:29.168461 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:29.168591 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:29.172249 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:29.172254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:29.172257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:29.172291 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:29.174316 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:29.175112 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:29.175118 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:29.175120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:29.175229 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:29.175801 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:29.175808 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:29.175810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:29.176078 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:29.176182 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:29.176981 (+ 799us) server_negotiation.cc:299] Negotiation successful
0806 00:56:29.177151 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 00:56:29.179740 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:29.160517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:45213 (local address 127.27.77.194:38611)
0806 00:56:29.161196 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:29.161268 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:29.162740 (+ 1472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:29.163771 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:29.163781 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:29.164198 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:29.164955 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:29.164981 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:29.166282 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:29.166287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:29.167909 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:29.167924 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:29.168837 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:29.168842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:29.169076 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:29.169837 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:29.169861 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:29.171864 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:29.174496 (+ 2632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:29.174504 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:29.174524 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:29.174951 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:29.175383 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:29.175387 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:29.175389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:29.175659 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:29.176265 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:29.176273 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:29.176653 (+ 380us) client_negotiation.cc:769] Sending connection context
0806 00:56:29.178081 (+ 1428us) client_negotiation.cc:240] Negotiation successful
0806 00:56:29.178498 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":149,"threads_started":1}
I20260806 00:56:29.181579 13011 heartbeater.cc:347] Connected to a master server at 127.27.77.254:45213
I20260806 00:56:29.181886 13011 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:29.182576 13011 heartbeater.cc:511] Master 127.27.77.254:45213 requested a full tablet report, sending...
W20260806 00:56:29.184567 13011 heartbeater.cc:644] Failed to heartbeat to 127.27.77.254:45213 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 00:56:29.184705 13011 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:29.184894 13011 heartbeater.cc:511] Master 127.27.77.254:45213 requested a full tablet report, sending...
I20260806 00:56:29.186064 13011 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:29.186262 13011 heartbeater.cc:511] Master 127.27.77.254:45213 requested a full tablet report, sending...
W20260806 00:56:29.187225 13011 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:56:29.321477 13015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:29.321939 13015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:29.322079 13015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:29.327826 13015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:29.327914 13015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:29.328022 13015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:56:29.334549 13015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:35045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.77.212:47485/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:41199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=40529
--tserver_master_addrs=127.27.77.254:45213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:29.336268 13015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:29.337349 13015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:29.347117 13022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:29.347576 13023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:29.348022 13025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:29.348129 13015 server_base.cc:1034] running on GCE node
I20260806 00:56:29.348681 13015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:29.349447 13015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:29.350703 13015 hybrid_clock.cc:648] HybridClock initialized: now 1785977789350664 us; error 51 us; skew 500 ppm
Aug 06 00:56:29 dist-test-slave-sb2z krb5kdc[11294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977789, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:29.354480 13015 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:56:29.355899 13015 webserver.cc:460] Webserver started at http://127.27.77.195:40529/ using document root <none> and password file <none>
I20260806 00:56:29.356695 13015 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:29.356837 13015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 00:56:29 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977789, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:56:29.383761 13015 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:56:29.387427 13035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:29.389322 13015 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:56:29.389473 13015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "468ac47b21ee434a84e9c56e407b4652"
format_stamp: "Formatted at 2026-08-06 00:56:28 on dist-test-slave-sb2z"
server_key: "6d0f57acd8423229d4f91d67dce38e42"
server_key_iv: "cdba438a893e221b0e2d48b0ab659464"
server_key_version: "kuduclusterkey@0"
I20260806 00:56:29.390205 13015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:29.405037 13015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:29.408859 13015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:29.409073 13015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:29.409832 13015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:29.410939 13015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:29.411043 13015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:29.411132 13015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:29.411196 13015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:29.423633 13015 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:41199
I20260806 00:56:29.423897 13178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:41199 every 8 connection(s)
I20260806 00:56:29.425529 13015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:56:29.434710 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 13015
Aug 06 00:56:29 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977789, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:29.449180 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:29.429961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:45213 (local address 127.27.77.195:37043)
0806 00:56:29.430592 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:29.430662 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:29.431927 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:29.432631 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:29.432641 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:29.433050 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:29.433906 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:29.433932 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:29.435050 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:29.435054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:29.439299 (+ 4245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:29.439315 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:29.440113 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:29.440118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:29.440321 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:29.441032 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:29.441056 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:29.442771 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:29.445221 (+ 2450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:29.445229 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:29.445250 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:29.445662 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:29.446061 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:29.446065 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:29.446067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:29.446283 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:29.446861 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:29.446870 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:29.447255 (+ 385us) client_negotiation.cc:769] Sending connection context
0806 00:56:29.447507 (+ 252us) client_negotiation.cc:240] Negotiation successful
0806 00:56:29.447926 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":144,"threads_started":1}
I20260806 00:56:29.449990 12845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:29.429432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:37043 (local address 127.27.77.254:45213)
0806 00:56:29.429615 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:29.429621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:29.430812 (+ 1191us) server_negotiation.cc:407] Connection header received
0806 00:56:29.432219 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:29.432223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:29.432285 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:29.432387 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:29.434140 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:29.434909 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:29.439505 (+ 4596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:29.439923 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:29.440054 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:29.442981 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:29.442986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:29.442989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:29.443022 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:29.445055 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:29.445801 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:29.445805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:29.445807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:29.445913 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:29.446417 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:29.446421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:29.446423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:29.446683 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:29.446794 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:29.449638 (+ 2844us) server_negotiation.cc:299] Negotiation successful
0806 00:56:29.449797 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 00:56:29.452087 13179 heartbeater.cc:347] Connected to a master server at 127.27.77.254:45213
I20260806 00:56:29.452385 13179 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:29.453076 13179 heartbeater.cc:511] Master 127.27.77.254:45213 requested a full tablet report, sending...
W20260806 00:56:29.455133 13179 heartbeater.cc:644] Failed to heartbeat to 127.27.77.254:45213 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 00:56:29.455286 13179 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:29.455497 13179 heartbeater.cc:511] Master 127.27.77.254:45213 requested a full tablet report, sending...
I20260806 00:56:29.456544 13185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:29.435563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37452 (local address 127.27.77.254:45213)
0806 00:56:29.435856 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:29.435860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:29.439852 (+ 3992us) server_negotiation.cc:407] Connection header received
0806 00:56:29.439980 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:29.439983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:29.440042 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:29.440140 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:29.447685 (+ 7545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:29.448538 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:29.449922 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:29.450304 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:29.450379 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:29.452400 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:29.452404 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:29.452407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:29.452434 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:29.454311 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:29.454870 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:29.454874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:29.454876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:29.454967 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:29.455317 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:29.455320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:29.455322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:29.455603 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:29.455829 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:29.456221 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 00:56:29.456358 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":102,"threads_started":1}
I20260806 00:56:29.456575 13179 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:29.456768 13179 heartbeater.cc:511] Master 127.27.77.254:45213 requested a full tablet report, sending...
W20260806 00:56:29.457760 13179 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 00:56:29 dist-test-slave-sb2z krb5kdc[11294](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785977789, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:29.901070 12841 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:29.901430 12841 heartbeater.cc:511] Master 127.27.77.254:45213 requested a full tablet report, sending...
I20260806 00:56:29.902776 12581 ts_manager.cc:194] Registered new tserver with Master: 45041559aa2648eeac724b33ae61b165 (127.27.77.193:45415)
I20260806 00:56:29.904774 12581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:37809
I20260806 00:56:30.187479 13011 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:30.187863 13011 heartbeater.cc:511] Master 127.27.77.254:45213 requested a full tablet report, sending...
I20260806 00:56:30.188654 12581 ts_manager.cc:194] Registered new tserver with Master: 00d0abd0377648f98771bb0d9b281677 (127.27.77.194:35989)
I20260806 00:56:30.189284 12581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:38611
Aug 06 00:56:30 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977789, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
I20260806 00:56:30.458019 13179 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:30.458360 13179 heartbeater.cc:511] Master 127.27.77.254:45213 requested a full tablet report, sending...
I20260806 00:56:30.459262 12581 ts_manager.cc:194] Registered new tserver with Master: 468ac47b21ee434a84e9c56e407b4652 (127.27.77.195:41199)
I20260806 00:56:30.460058 12581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:37043
I20260806 00:56:30.468272 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:56:30.479251 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:30.470185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37462 (local address 127.27.77.254:45213)
0806 00:56:30.470505 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:30.470510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:30.470624 (+ 114us) server_negotiation.cc:407] Connection header received
0806 00:56:30.470802 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:30.470805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:30.470860 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:30.470969 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:30.472044 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.472974 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.473908 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.474163 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.474278 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:30.475793 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:30.475796 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:30.475799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:30.475827 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:30.477420 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:30.477875 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:30.477879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:30.477881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:30.477972 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:30.478283 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:30.478286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:30.478288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:30.478486 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:30.478642 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:30.478892 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 00:56:30.479031 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":80,"threads_started":1}
I20260806 00:56:30.483238 12581 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:56:30.487030 12581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:56:30.509473 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:30.501293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:45415 (local address 127.0.0.1:36466)
0806 00:56:30.502879 (+ 1586us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:30.502893 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:30.502984 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:30.504221 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:30.504227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:30.504254 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:30.504416 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:30.504422 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.505529 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.505535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:30.506532 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:30.506550 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.508600 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.508604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:30.508722 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:30.509059 (+ 337us) client_negotiation.cc:769] Sending connection context
0806 00:56:30.509171 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 00:56:30.509313 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1514,"thread_start_us":134,"threads_started":1}
I20260806 00:56:30.509475 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:30.501773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36466 (local address 127.27.77.193:45415)
0806 00:56:30.503479 (+ 1706us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:30.503485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:30.503512 (+ 27us) server_negotiation.cc:407] Connection header received
0806 00:56:30.503617 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:30.503623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:30.503862 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:30.504560 (+ 698us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:30.504673 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.505403 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.507100 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.508423 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.508597 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:30.508765 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:30.509213 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 00:56:30.509315 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1578,"thread_start_us":132,"threads_started":1}
I20260806 00:56:30.513302 12757 tablet_service.cc:1467] Processing CreateTablet for tablet 1ef9de623ac34bfcadec6affcbdbe6c4 (DEFAULT_TABLE table=test-table [id=4a38dd40b64d47bd9a2a6deed642047a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:56:30.514523 12757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ef9de623ac34bfcadec6affcbdbe6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:30.516528 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:30.502626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:41199 (local address 127.0.0.1:44310)
0806 00:56:30.509928 (+ 7302us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:30.509943 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:30.510028 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:30.510770 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:30.510774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:30.510792 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:30.511770 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:30.511778 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.513138 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.513141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:30.514205 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:30.514218 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.515303 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.515307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:30.515389 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:30.515420 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 00:56:30.515475 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 00:56:30.516386 (+ 911us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7200,"thread_start_us":104,"threads_started":1}
I20260806 00:56:30.517231 13218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:30.502951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44310 (local address 127.27.77.195:41199)
0806 00:56:30.503259 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:30.503267 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:30.510148 (+ 6881us) server_negotiation.cc:407] Connection header received
0806 00:56:30.510245 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:30.510250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:30.510448 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:30.510601 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:30.511917 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.512998 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.514368 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.515160 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.516747 (+ 1587us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:30.516923 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:30.516974 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 00:56:30.517070 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":112,"threads_started":1}
I20260806 00:56:30.520447 13083 tablet_service.cc:1467] Processing CreateTablet for tablet 1ef9de623ac34bfcadec6affcbdbe6c4 (DEFAULT_TABLE table=test-table [id=4a38dd40b64d47bd9a2a6deed642047a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:56:30.522667 13083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ef9de623ac34bfcadec6affcbdbe6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:30.525251 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:30.499943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:35989 (local address 127.0.0.1:38288)
0806 00:56:30.501725 (+ 1782us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:30.501781 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:30.501945 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:30.508884 (+ 6939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:30.508887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:30.508905 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:30.508980 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:30.508984 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.517099 (+ 8115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.517103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:30.518376 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:30.518388 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.524752 (+ 6364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.524768 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:30.524882 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:30.524914 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 00:56:30.524972 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 00:56:30.525087 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1573,"thread_start_us":214,"threads_started":1}
I20260806 00:56:30.526264 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:30.500091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38288 (local address 127.27.77.194:35989)
0806 00:56:30.508298 (+ 8207us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:30.508306 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:30.508334 (+ 28us) server_negotiation.cc:407] Connection header received
0806 00:56:30.508416 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:30.508422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:30.508683 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:30.508881 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:30.509051 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.510271 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.518567 (+ 8296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.519473 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.525718 (+ 6245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:30.525908 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:30.525970 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 00:56:30.526077 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8073,"thread_start_us":113,"threads_started":1}
I20260806 00:56:30.526436 13219 tablet_bootstrap.cc:492] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 45041559aa2648eeac724b33ae61b165: Bootstrap starting.
I20260806 00:56:30.529031 13219 tablet_bootstrap.cc:654] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 45041559aa2648eeac724b33ae61b165: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:30.530099 13219 log.cc:824] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 45041559aa2648eeac724b33ae61b165: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:30.529690 12929 tablet_service.cc:1467] Processing CreateTablet for tablet 1ef9de623ac34bfcadec6affcbdbe6c4 (DEFAULT_TABLE table=test-table [id=4a38dd40b64d47bd9a2a6deed642047a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:56:30.530889 12929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ef9de623ac34bfcadec6affcbdbe6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:30.532963 13219 tablet_bootstrap.cc:492] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 45041559aa2648eeac724b33ae61b165: No bootstrap required, opened a new log
I20260806 00:56:30.533179 13219 ts_tablet_manager.cc:1397] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 45041559aa2648eeac724b33ae61b165: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:56:30.537309 13219 raft_consensus.cc:348] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 45041559aa2648eeac724b33ae61b165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d0abd0377648f98771bb0d9b281677" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35989 } } peers { permanent_uuid: "45041559aa2648eeac724b33ae61b165" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 45415 } } peers { permanent_uuid: "468ac47b21ee434a84e9c56e407b4652" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41199 } }
I20260806 00:56:30.537755 13219 raft_consensus.cc:374] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 45041559aa2648eeac724b33ae61b165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:30.537871 13219 raft_consensus.cc:729] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 45041559aa2648eeac724b33ae61b165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45041559aa2648eeac724b33ae61b165, State: Initialized, Role: FOLLOWER
I20260806 00:56:30.538460 13219 consensus_queue.cc:260] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 45041559aa2648eeac724b33ae61b165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d0abd0377648f98771bb0d9b281677" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35989 } } peers { permanent_uuid: "45041559aa2648eeac724b33ae61b165" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 45415 } } peers { permanent_uuid: "468ac47b21ee434a84e9c56e407b4652" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41199 } }
I20260806 00:56:30.539764 13219 ts_tablet_manager.cc:1428] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 45041559aa2648eeac724b33ae61b165: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 00:56:30.540190 12841 heartbeater.cc:503] Master 127.27.77.254:45213 was elected leader, sending a full tablet report...
I20260806 00:56:30.546198 13225 tablet_bootstrap.cc:492] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677: Bootstrap starting.
I20260806 00:56:30.548816 13225 tablet_bootstrap.cc:654] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:30.549644 13225 log.cc:824] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:30.552404 13225 tablet_bootstrap.cc:492] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677: No bootstrap required, opened a new log
I20260806 00:56:30.552639 13225 ts_tablet_manager.cc:1397] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:56:30.556066 13225 raft_consensus.cc:348] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d0abd0377648f98771bb0d9b281677" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35989 } } peers { permanent_uuid: "45041559aa2648eeac724b33ae61b165" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 45415 } } peers { permanent_uuid: "468ac47b21ee434a84e9c56e407b4652" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41199 } }
I20260806 00:56:30.556421 13225 raft_consensus.cc:374] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:30.556540 13225 raft_consensus.cc:729] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d0abd0377648f98771bb0d9b281677, State: Initialized, Role: FOLLOWER
I20260806 00:56:30.557149 13225 consensus_queue.cc:260] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d0abd0377648f98771bb0d9b281677" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35989 } } peers { permanent_uuid: "45041559aa2648eeac724b33ae61b165" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 45415 } } peers { permanent_uuid: "468ac47b21ee434a84e9c56e407b4652" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41199 } }
I20260806 00:56:30.558372 13225 ts_tablet_manager.cc:1428] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:56:30.558817 13011 heartbeater.cc:503] Master 127.27.77.254:45213 was elected leader, sending a full tablet report...
I20260806 00:56:30.561272 13226 tablet_bootstrap.cc:492] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652: Bootstrap starting.
I20260806 00:56:30.564066 13226 tablet_bootstrap.cc:654] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:30.565106 13226 log.cc:824] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:30.568109 13226 tablet_bootstrap.cc:492] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652: No bootstrap required, opened a new log
I20260806 00:56:30.568342 13226 ts_tablet_manager.cc:1397] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260806 00:56:30.572305 13226 raft_consensus.cc:348] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d0abd0377648f98771bb0d9b281677" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35989 } } peers { permanent_uuid: "45041559aa2648eeac724b33ae61b165" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 45415 } } peers { permanent_uuid: "468ac47b21ee434a84e9c56e407b4652" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41199 } }
I20260806 00:56:30.572701 13226 raft_consensus.cc:374] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:30.572814 13226 raft_consensus.cc:729] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468ac47b21ee434a84e9c56e407b4652, State: Initialized, Role: FOLLOWER
I20260806 00:56:30.573427 13226 consensus_queue.cc:260] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d0abd0377648f98771bb0d9b281677" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35989 } } peers { permanent_uuid: "45041559aa2648eeac724b33ae61b165" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 45415 } } peers { permanent_uuid: "468ac47b21ee434a84e9c56e407b4652" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41199 } }
I20260806 00:56:30.574649 13226 ts_tablet_manager.cc:1428] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:56:30.575165 13179 heartbeater.cc:503] Master 127.27.77.254:45213 was elected leader, sending a full tablet report...
Aug 06 00:56:30 dist-test-slave-sb2z krb5kdc[11294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977789, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for HTTP/127.27.77.212@KRBTEST.COM
W20260806 00:56:30.623364 12842 tablet.cc:2369] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 45041559aa2648eeac724b33ae61b165: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:56:30.659170 13012 tablet.cc:2369] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:56:30.665515 13233 raft_consensus.cc:482] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:56:30.665781 13233 raft_consensus.cc:504] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d0abd0377648f98771bb0d9b281677" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35989 } } peers { permanent_uuid: "45041559aa2648eeac724b33ae61b165" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 45415 } } peers { permanent_uuid: "468ac47b21ee434a84e9c56e407b4652" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41199 } }
I20260806 00:56:30.667338 13233 leader_election.cc:288] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00d0abd0377648f98771bb0d9b281677 (127.27.77.194:35989), 45041559aa2648eeac724b33ae61b165 (127.27.77.193:45415)
I20260806 00:56:30.673161 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:30.667884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:56451 (local address 127.27.77.193:45415)
0806 00:56:30.668029 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:30.668034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:30.668559 (+ 525us) server_negotiation.cc:407] Connection header received
0806 00:56:30.668768 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:30.668772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:30.668826 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:30.668877 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:30.669333 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.669962 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.671700 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.672313 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.672418 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:30.672475 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:30.672897 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 00:56:30.672990 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:56:30.673275 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:30.667552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:35989 (local address 127.27.77.195:42805)
0806 00:56:30.668048 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:30.668065 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:30.668196 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:30.668691 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:30.668695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:30.668719 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:30.668888 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:30.668896 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.670159 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.670163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:30.671623 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:30.671637 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.672770 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.672773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:30.672875 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:30.672900 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 00:56:30.672974 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 00:56:30.673077 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":1,"thread_start_us":151,"threads_started":1}
I20260806 00:56:30.673637 13237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:30.667704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:45415 (local address 127.27.77.195:56451)
0806 00:56:30.668465 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:30.668478 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:30.668634 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:30.669037 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:30.669039 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:30.669054 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:30.669212 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:30.669217 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.670085 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.670099 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:30.671505 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:30.671518 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.672622 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.672625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:30.672734 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:30.672768 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 00:56:30.673394 (+ 626us) client_negotiation.cc:240] Negotiation successful
0806 00:56:30.673497 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":66,"thread_start_us":88,"threads_started":1}
I20260806 00:56:30.673683 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:30.668158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:42805 (local address 127.27.77.194:35989)
0806 00:56:30.668307 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:30.668312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:30.668330 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:56:30.668395 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:30.668398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:30.668451 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:30.668542 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:30.669044 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.669948 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.671791 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.672497 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.672610 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:30.672664 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:30.673458 (+ 794us) server_negotiation.cc:299] Negotiation successful
0806 00:56:30.673554 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:56:30.674418 12783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef9de623ac34bfcadec6affcbdbe6c4" candidate_uuid: "468ac47b21ee434a84e9c56e407b4652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45041559aa2648eeac724b33ae61b165" is_pre_election: true
I20260806 00:56:30.674788 12783 raft_consensus.cc:2454] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 45041559aa2648eeac724b33ae61b165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468ac47b21ee434a84e9c56e407b4652 in term 0.
I20260806 00:56:30.675376 13038 leader_election.cc:302] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45041559aa2648eeac724b33ae61b165, 468ac47b21ee434a84e9c56e407b4652; no voters:
I20260806 00:56:30.675716 13233 raft_consensus.cc:2781] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:56:30.675825 13233 raft_consensus.cc:482] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:56:30.675873 13233 raft_consensus.cc:3037] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:30.677361 13233 raft_consensus.cc:504] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d0abd0377648f98771bb0d9b281677" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35989 } } peers { permanent_uuid: "45041559aa2648eeac724b33ae61b165" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 45415 } } peers { permanent_uuid: "468ac47b21ee434a84e9c56e407b4652" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41199 } }
I20260806 00:56:30.677824 13233 leader_election.cc:288] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [CANDIDATE]: Term 1 election: Requested vote from peers 00d0abd0377648f98771bb0d9b281677 (127.27.77.194:35989), 45041559aa2648eeac724b33ae61b165 (127.27.77.193:45415)
I20260806 00:56:30.678324 12783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef9de623ac34bfcadec6affcbdbe6c4" candidate_uuid: "468ac47b21ee434a84e9c56e407b4652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45041559aa2648eeac724b33ae61b165"
I20260806 00:56:30.678483 12783 raft_consensus.cc:3037] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 45041559aa2648eeac724b33ae61b165 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:30.678717 12931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef9de623ac34bfcadec6affcbdbe6c4" candidate_uuid: "468ac47b21ee434a84e9c56e407b4652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d0abd0377648f98771bb0d9b281677" is_pre_election: true
I20260806 00:56:30.679066 12931 raft_consensus.cc:2454] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468ac47b21ee434a84e9c56e407b4652 in term 0.
W20260806 00:56:30.679719 13180 tablet.cc:2369] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:56:30.680164 12783 raft_consensus.cc:2454] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 45041559aa2648eeac724b33ae61b165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468ac47b21ee434a84e9c56e407b4652 in term 1.
I20260806 00:56:30.679441 12948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef9de623ac34bfcadec6affcbdbe6c4" candidate_uuid: "468ac47b21ee434a84e9c56e407b4652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d0abd0377648f98771bb0d9b281677"
I20260806 00:56:30.680343 12948 raft_consensus.cc:3037] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:30.680507 13038 leader_election.cc:302] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45041559aa2648eeac724b33ae61b165, 468ac47b21ee434a84e9c56e407b4652; no voters:
I20260806 00:56:30.680812 13233 raft_consensus.cc:2781] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:56:30.681115 13233 raft_consensus.cc:686] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [term 1 LEADER]: Becoming Leader. State: Replica: 468ac47b21ee434a84e9c56e407b4652, State: Running, Role: LEADER
I20260806 00:56:30.681507 13233 consensus_queue.cc:237] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d0abd0377648f98771bb0d9b281677" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35989 } } peers { permanent_uuid: "45041559aa2648eeac724b33ae61b165" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 45415 } } peers { permanent_uuid: "468ac47b21ee434a84e9c56e407b4652" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41199 } }
I20260806 00:56:30.681723 12948 raft_consensus.cc:2454] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468ac47b21ee434a84e9c56e407b4652 in term 1.
I20260806 00:56:30.685549 12581 catalog_manager.cc:5495] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468ac47b21ee434a84e9c56e407b4652 (127.27.77.195). New cstate: current_term: 1 leader_uuid: "468ac47b21ee434a84e9c56e407b4652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d0abd0377648f98771bb0d9b281677" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45041559aa2648eeac724b33ae61b165" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 45415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468ac47b21ee434a84e9c56e407b4652" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 41199 } health_report { overall_health: HEALTHY } } }
I20260806 00:56:30.715611 13218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:30.709877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44318 (local address 127.27.77.195:41199)
0806 00:56:30.710173 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:30.710182 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:30.710202 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:56:30.710315 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:30.710322 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:30.710423 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:30.710555 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:56:30.711001 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.711968 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.713348 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:30.713729 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:30.713848 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:30.714358 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:56:30.714964 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:56:30.715110 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:30.715316 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 00:56:30.715388 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":161}
I20260806 00:56:30.729912 12783 raft_consensus.cc:1261] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 45041559aa2648eeac724b33ae61b165 [term 1 FOLLOWER]: Refusing update from remote peer 468ac47b21ee434a84e9c56e407b4652: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:56:30.730005 12948 raft_consensus.cc:1261] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677 [term 1 FOLLOWER]: Refusing update from remote peer 468ac47b21ee434a84e9c56e407b4652: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:56:30.730703 13233 consensus_queue.cc:1035] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45041559aa2648eeac724b33ae61b165" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 45415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:56:30.731616 13233 consensus_queue.cc:1035] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00d0abd0377648f98771bb0d9b281677" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 35989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:56:30.742561 13243 mvcc.cc:205] Tried to move back new op lower bound from 7315365030818803712 to 7315365030635651072. Current Snapshot: MvccSnapshot[applied={T|T < 7315365030818803712}]
I20260806 00:56:30.750764 13245 mvcc.cc:205] Tried to move back new op lower bound from 7315365030818803712 to 7315365030635651072. Current Snapshot: MvccSnapshot[applied={T|T < 7315365030818803712}]
I20260806 00:56:30.754072 13244 mvcc.cc:205] Tried to move back new op lower bound from 7315365030818803712 to 7315365030635651072. Current Snapshot: MvccSnapshot[applied={T|T < 7315365030818803712}]
I20260806 00:56:30.761914 12581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:56:30.762131 12581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:56:30.765298 12581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb3cfca399c14b1d95780b5fdff28e83: Sending DeleteTablet for 3 replicas of tablet 1ef9de623ac34bfcadec6affcbdbe6c4
I20260806 00:56:30.766639 12757 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ef9de623ac34bfcadec6affcbdbe6c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:56:30 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:36466
I20260806 00:56:30.767261 12929 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ef9de623ac34bfcadec6affcbdbe6c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:56:30 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:38288
I20260806 00:56:30.767292 13083 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ef9de623ac34bfcadec6affcbdbe6c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:56:30 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:44310
I20260806 00:56:30.767892 13255 tablet_replica.cc:330] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652: stopping tablet replica
I20260806 00:56:30.768136 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 12632
I20260806 00:56:30.768807 13255 raft_consensus.cc:2229] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:56:30.769220 13255 raft_consensus.cc:2258] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:56:30.771510 13255 ts_tablet_manager.cc:1905] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:56:30.775352 13255 ts_tablet_manager.cc:1918] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:56:30.775484 13255 log.cc:1196] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/1ef9de623ac34bfcadec6affcbdbe6c4
I20260806 00:56:30.775988 13255 ts_tablet_manager.cc:1939] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 468ac47b21ee434a84e9c56e407b4652: Deleting consensus metadata
I20260806 00:56:30.777861 12562 catalog_manager.cc:4841] TS 468ac47b21ee434a84e9c56e407b4652 (127.27.77.195:41199): tablet 1ef9de623ac34bfcadec6affcbdbe6c4 (table test-table [id=4a38dd40b64d47bd9a2a6deed642047a]) successfully deleted
W20260806 00:56:30.788516 12564 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:45415 (error 108)
I20260806 00:56:30.788962 13254 tablet_replica.cc:330] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677: stopping tablet replica
I20260806 00:56:30.789453 13254 raft_consensus.cc:2229] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:56:30.789584 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 12844
I20260806 00:56:30.790015 13254 raft_consensus.cc:2258] T 1ef9de623ac34bfcadec6affcbdbe6c4 P 00d0abd0377648f98771bb0d9b281677 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:56:30.791808 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:30.789977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:45415 (local address 127.0.0.1:36480)
0806 00:56:30.791582 (+ 1605us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:30.791700 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:45415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1440}
W20260806 00:56:30.792268 12564 catalog_manager.cc:4584] TS 45041559aa2648eeac724b33ae61b165 (127.27.77.193:45415): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ef9de623ac34bfcadec6affcbdbe6c4: Network error: Client connection negotiation failed: client connection to 127.27.77.193:45415: connect: Connection refused (error 111)
I20260806 00:56:30.801995 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 13015
I20260806 00:56:30.808692 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:30.808309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:35989 (local address 127.0.0.1:38294)
0806 00:56:30.808530 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:30.808591 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.194:35989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260806 00:56:30.815359 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 12535
I20260806 00:56:30.829972 27959 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 00:56:31.188256 27959 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 00:56:31.188385 27959 mini_ranger.cc:66] Stopping Ranger...
I20260806 00:56:31.543903 27959 mini_ranger.cc:68] Stopped Ranger
2026-08-06 00:56:31.544 UTC [11330] LOG: received smart shutdown request
2026-08-06 00:56:31.548 UTC [11330] LOG: background worker "logical replication launcher" (PID 11339) exited with exit code 1
2026-08-06 00:56:31.549 UTC [11334] LOG: shutting down
2026-08-06 00:56:31.742 UTC [11330] LOG: database system is shut down
2026-08-06T00:56:31Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52816 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:56:31 dist-test-slave-sb2z krb5kdc[13268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:56:31 dist-test-slave-sb2z krb5kdc[13268](info): set up 2 sockets
Aug 06 00:56:31 dist-test-slave-sb2z krb5kdc[13268](info): commencing operation
krb5kdc: starting...
W20260806 00:56:33.080651 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:56:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:56:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:56:33.214264 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:45947
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:45947,127.27.77.253:32861,127.27.77.252:42043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:33.340626 13285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:33.340945 13285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:33.341022 13285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:33.345029 13285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:56:33.345109 13285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:33.345147 13285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:33.345183 13285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:56:33.345216 13285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:56:33.350494 13285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:45947,127.27.77.253:32861,127.27.77.252:42043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:45947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:33.351642 13285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:33.352743 13285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:33.361871 13290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:33.361927 13285 server_base.cc:1034] running on GCE node
W20260806 00:56:33.361824 13295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:33.361825 13291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:33.362876 13285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:33.364006 13285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:33.365185 13285 hybrid_clock.cc:648] HybridClock initialized: now 1785977793365156 us; error 44 us; skew 500 ppm
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:33.368172 13285 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:56:33.369033 13285 webserver.cc:460] Webserver started at http://127.27.77.254:42105/ using document root <none> and password file <none>
I20260806 00:56:33.369556 13285 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:33.369613 13285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:33.369776 13285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:56:33.371901 13285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "30bc0c578ec145f586991b480494d0b8"
format_stamp: "Formatted at 2026-08-06 00:56:33 on dist-test-slave-sb2z"
server_key: "89a96e98ca307f704a00ef0b29ceb67b"
server_key_iv: "cb7e5a38dced788050452e67e9b24952"
server_key_version: "encryptionkey@0"
I20260806 00:56:33.372423 13285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "30bc0c578ec145f586991b480494d0b8"
format_stamp: "Formatted at 2026-08-06 00:56:33 on dist-test-slave-sb2z"
server_key: "89a96e98ca307f704a00ef0b29ceb67b"
server_key_iv: "cb7e5a38dced788050452e67e9b24952"
server_key_version: "encryptionkey@0"
I20260806 00:56:33.376183 13285 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:56:33.378677 13303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:33.380036 13285 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:56:33.380148 13285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "30bc0c578ec145f586991b480494d0b8"
format_stamp: "Formatted at 2026-08-06 00:56:33 on dist-test-slave-sb2z"
server_key: "89a96e98ca307f704a00ef0b29ceb67b"
server_key_iv: "cb7e5a38dced788050452e67e9b24952"
server_key_version: "encryptionkey@0"
I20260806 00:56:33.380236 13285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:33.411271 13285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:33.414791 13285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:33.414953 13285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:33.422355 13285 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:45947
I20260806 00:56:33.422468 13371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:45947 every 8 connection(s)
I20260806 00:56:33.423364 13285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:56:33.426342 13372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:33.430374 13372 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:56:33.432516 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 13285
I20260806 00:56:33.432657 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:56:33.432963 27959 external_mini_cluster.cc:1428] Setting key a38344b2e01a555a602ac52103e49c51
WARNING: no policy specified for kudu/127.27.77.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.253@KRBTEST.COM" created.
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:33.452116 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.432952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46208 (local address 127.27.77.254:45947)
0806 00:56:33.433490 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:33.433510 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:33.433914 (+ 404us) server_negotiation.cc:407] Connection header received
0806 00:56:33.435648 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:33.435658 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:33.435927 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:33.436148 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:33.438678 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.439725 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.440871 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.441300 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.441499 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.446821 (+ 5322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:33.446827 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:33.446839 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:33.446863 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:33.448786 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.449301 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.449307 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.449309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.449435 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.449763 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.449766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.449768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.450162 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:33.450303 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:33.450687 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 00:56:33.450975 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":341,"thread_start_us":157,"threads_started":1}
I20260806 00:56:33.452116 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.432952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:45947 (local address 127.0.0.1:46208)
0806 00:56:33.433717 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.433779 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:33.435209 (+ 1430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:33.436172 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:33.436177 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:33.436545 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:33.436827 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.436841 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.439861 (+ 3020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.439865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.440721 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.440731 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.441475 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.441480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.441597 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.442272 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:33.442291 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:33.446644 (+ 4353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:33.448929 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.448939 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.448949 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.449182 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.449535 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.449538 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.449541 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.449658 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.450285 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:33.450292 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:33.450440 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 00:56:33.450612 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 00:56:33.451028 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":108,"spinlock_wait_cycles":59136,"thread_start_us":199,"threads_started":1}
I20260806 00:56:33.454926 13372 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:56:33.455931 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.455490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:32861 (local address 127.0.0.1:40640)
0806 00:56:33.455778 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.455859 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203}
W20260806 00:56:33.456202 13305 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.77.253:32861: connect: Connection refused (error 111)
I20260806 00:56:33.457144 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.456872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:32861 (local address 127.0.0.1:40650)
0806 00:56:33.457035 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.457083 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 00:56:33.457304 13372 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.253:32861: Network error: Client connection negotiation failed: client connection to 127.27.77.253:32861: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.253.keytab.
Entry for principal kudu/127.27.77.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.253.keytab.
I20260806 00:56:33.463801 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.253:32861
--webserver_interface=127.27.77.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.253.keytab
--principal=kudu/127.27.77.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:45947,127.27.77.253:32861,127.27.77.252:42043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:56:33.506656 13372 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } attempt: 1
I20260806 00:56:33.507714 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.507339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:32861 (local address 127.0.0.1:40662)
0806 00:56:33.507549 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.507625 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 00:56:33.508579 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.508351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:32861 (local address 127.0.0.1:40674)
0806 00:56:33.508475 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.508522 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 00:56:33.508811 13372 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.253:32861: Network error: Client connection negotiation failed: client connection to 127.27.77.253:32861: connect: Connection refused (error 111)
I20260806 00:56:33.576987 13372 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } attempt: 2
I20260806 00:56:33.577900 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.577597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:32861 (local address 127.0.0.1:40688)
0806 00:56:33.577757 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.577823 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 00:56:33.578678 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.578447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:32861 (local address 127.0.0.1:40704)
0806 00:56:33.578568 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.578624 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 00:56:33.578816 13372 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.253:32861: Network error: Client connection negotiation failed: client connection to 127.27.77.253:32861: connect: Connection refused (error 111)
W20260806 00:56:33.584833 13381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:33.585114 13381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:33.585218 13381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:33.589222 13381 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:56:33.589306 13381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:33.589357 13381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:33.589399 13381 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:56:33.589455 13381 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:56:33.594856 13381 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:45947,127.27.77.253:32861,127.27.77.252:42043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.253.keytab
--principal=kudu/127.27.77.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.253:32861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.77.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:33.596025 13381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:33.597148 13381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:33.605669 13387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:33.605680 13389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:33.605700 13386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:33.606529 13381 server_base.cc:1034] running on GCE node
I20260806 00:56:33.607187 13381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:33.608372 13381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:33.609543 13381 hybrid_clock.cc:648] HybridClock initialized: now 1785977793609516 us; error 41 us; skew 500 ppm
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:33.612576 13381 init.cc:377] Logged in from keytab as kudu/127.27.77.253@KRBTEST.COM (short username kudu)
I20260806 00:56:33.613399 13381 webserver.cc:460] Webserver started at http://127.27.77.253:45273/ using document root <none> and password file <none>
I20260806 00:56:33.613904 13381 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:33.613965 13381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:33.614142 13381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:56:33.616156 13381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/data/instance:
uuid: "769d0f1c6728420d9e92e3f35c8a198b"
format_stamp: "Formatted at 2026-08-06 00:56:33 on dist-test-slave-sb2z"
server_key: "491b72c82d49d79472bf3b6bb7a9822c"
server_key_iv: "72becdc80392c8df80ac558fa9994cf8"
server_key_version: "encryptionkey@0"
I20260806 00:56:33.616667 13381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/wal/instance:
uuid: "769d0f1c6728420d9e92e3f35c8a198b"
format_stamp: "Formatted at 2026-08-06 00:56:33 on dist-test-slave-sb2z"
server_key: "491b72c82d49d79472bf3b6bb7a9822c"
server_key_iv: "72becdc80392c8df80ac558fa9994cf8"
server_key_version: "encryptionkey@0"
I20260806 00:56:33.620317 13381 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:56:33.622785 13399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:33.624035 13381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:56:33.624174 13381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/wal
uuid: "769d0f1c6728420d9e92e3f35c8a198b"
format_stamp: "Formatted at 2026-08-06 00:56:33 on dist-test-slave-sb2z"
server_key: "491b72c82d49d79472bf3b6bb7a9822c"
server_key_iv: "72becdc80392c8df80ac558fa9994cf8"
server_key_version: "encryptionkey@0"
I20260806 00:56:33.624277 13381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:33.657311 13381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:33.660835 13381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:33.661068 13381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:33.668751 13381 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.253:32861
I20260806 00:56:33.668776 13462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.253:32861 every 8 connection(s)
I20260806 00:56:33.669741 13381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/data/info.pb
I20260806 00:56:33.669991 13372 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } attempt: 3
I20260806 00:56:33.672086 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 13381
I20260806 00:56:33.672231 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-1/wal/instance
I20260806 00:56:33.672533 27959 external_mini_cluster.cc:1428] Setting key 633158e20763fdbe589511419d83a806
I20260806 00:56:33.673396 13463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:33.678092 13463 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for kudu/127.27.77.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.77.252@KRBTEST.COM; defaulting to no policy
I20260806 00:56:33.685766 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.670582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:32861 (local address 127.0.0.1:40720)
0806 00:56:33.670757 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.670774 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:33.670876 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:33.673497 (+ 2621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:33.673501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:33.673727 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:33.673876 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.673886 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.675759 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.675763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.676603 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.676616 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.677802 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.677806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.677926 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.678567 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:33.678583 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:33.680531 (+ 1948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:33.683689 (+ 3158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.683693 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.683695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.683954 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.684365 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.684368 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.684370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.684467 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.685177 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:33.685182 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:33.685296 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 00:56:33.685422 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 00:56:33.685582 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 00:56:33.687057 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.671357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40720 (local address 127.27.77.253:32861)
0806 00:56:33.671906 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:33.671923 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:33.671963 (+ 40us) server_negotiation.cc:407] Connection header received
0806 00:56:33.672841 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:33.672862 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:33.673210 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:33.673583 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:33.674065 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.675599 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.676755 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.677646 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.677874 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.680834 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:33.680845 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:33.680866 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:33.680903 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:33.683560 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.684064 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.684074 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.684084 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.684262 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.684583 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.684588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.684594 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.685004 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:33.685170 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:33.685603 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 00:56:33.686232 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":184,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.252@KRBTEST.COM" created.
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.253@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:33.690723 13372 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:56:33.691606 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.691258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42043 (local address 127.0.0.1:55660)
0806 00:56:33.691413 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.691480 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 00:56:33.692435 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.692150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42043 (local address 127.0.0.1:55674)
0806 00:56:33.692303 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.692356 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260806 00:56:33.692705 13372 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:42043: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
I20260806 00:56:33.692809 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.680170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46212 (local address 127.27.77.254:45947)
0806 00:56:33.680305 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:33.680309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:33.680687 (+ 378us) server_negotiation.cc:407] Connection header received
0806 00:56:33.680874 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:33.680877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:33.680939 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:33.681030 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:33.681901 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.682954 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.684329 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.684696 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.684809 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.689057 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:33.689065 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:33.689068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:33.689090 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:33.690599 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.691185 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.691188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.691190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.691281 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.691661 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.691664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.691666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.691922 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:33.692033 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:33.692477 (+ 444us) server_negotiation.cc:299] Negotiation successful
0806 00:56:33.692622 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 00:56:33.692873 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.680133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:45947 (local address 127.0.0.1:46212)
0806 00:56:33.680552 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.680600 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:33.680777 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:33.681067 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:33.681073 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:33.681636 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:33.681771 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.681777 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.683069 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.683074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.684190 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.684204 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.684822 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.684826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.684950 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.685522 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:33.685535 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:33.688909 (+ 3374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:33.690739 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.690750 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.690759 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.691072 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.691403 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.691406 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.691408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.691563 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.692053 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:33.692062 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:33.692209 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 00:56:33.692364 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 00:56:33.692596 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":108,"threads_started":1}
I20260806 00:56:33.694010 13463 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.253@KRBTEST.COM for kudu/127.27.77.253@KRBTEST.COM
I20260806 00:56:33.704818 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.694470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:32861 (local address 127.0.0.1:40736)
0806 00:56:33.694581 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.694595 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:33.694679 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:33.695005 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:33.695007 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:33.695139 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:33.695231 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.695235 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.696353 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.696356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.696876 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.696885 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.697524 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.697526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.697676 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.698131 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:33.698141 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:33.701297 (+ 3156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:33.703160 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.703163 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.703165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.703431 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.703758 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.703760 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.703762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.703881 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.704320 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:33.704323 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:33.704413 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 00:56:33.704507 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 00:56:33.704610 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":39}
I20260806 00:56:33.704821 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.694622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40736 (local address 127.27.77.253:32861)
0806 00:56:33.694765 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:33.694772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:33.694789 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:56:33.694832 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:33.694837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:33.694891 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:33.694990 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:33.695349 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.696225 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.696997 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.697408 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.697571 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.701428 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:33.701431 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:33.701433 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:33.701454 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:33.703052 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.703519 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.703561 (+ 42us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.703563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.703644 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.703986 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.703988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.703989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.704175 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:33.704255 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:33.704580 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 00:56:33.704676 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":50}
I20260806 00:56:33.705801 13463 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.252.keytab.
Entry for principal kudu/127.27.77.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.252.keytab.
I20260806 00:56:33.706579 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.706285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42043 (local address 127.0.0.1:55676)
0806 00:56:33.706418 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.706513 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 00:56:33.706944 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.252:42043
--webserver_interface=127.27.77.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.252.keytab
--principal=kudu/127.27.77.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:45947,127.27.77.253:32861,127.27.77.252:42043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:33.707283 13402 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
I20260806 00:56:33.708325 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.708025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42043 (local address 127.0.0.1:55692)
0806 00:56:33.708176 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.708243 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 00:56:33.708559 13463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:42043: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
I20260806 00:56:33.723874 13372 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } attempt: 1
I20260806 00:56:33.724781 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.724478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42043 (local address 127.0.0.1:55696)
0806 00:56:33.724639 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.724700 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 00:56:33.725504 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.725316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42043 (local address 127.0.0.1:55710)
0806 00:56:33.725412 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.725456 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 00:56:33.725672 13372 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:42043: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
I20260806 00:56:33.757881 13463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } attempt: 1
I20260806 00:56:33.758903 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.758526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42043 (local address 127.0.0.1:55722)
0806 00:56:33.758722 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.758797 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 00:56:33.759742 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.759461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42043 (local address 127.0.0.1:55732)
0806 00:56:33.759610 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.759668 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 00:56:33.759912 13463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:42043: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
I20260806 00:56:33.800840 13372 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } attempt: 2
I20260806 00:56:33.801901 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.801502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42043 (local address 127.0.0.1:55734)
0806 00:56:33.801699 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.801782 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 00:56:33.802834 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.802583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42043 (local address 127.0.0.1:55750)
0806 00:56:33.802726 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.802779 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 00:56:33.803000 13372 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:42043: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
W20260806 00:56:33.824246 13470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:33.824493 13470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:33.824569 13470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 00:56:33.828119 13463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } attempt: 2
W20260806 00:56:33.829020 13470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
I20260806 00:56:33.829106 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.828710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42043 (local address 127.0.0.1:55764)
0806 00:56:33.828913 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.829003 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 00:56:33.829126 13470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:33.829161 13470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:33.829188 13470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:56:33.829210 13470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:56:33.830375 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.830098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42043 (local address 127.0.0.1:55770)
0806 00:56:33.830237 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.830302 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 00:56:33.830519 13463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:42043: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42043: connect: Connection refused (error 111)
I20260806 00:56:33.835026 13470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:45947,127.27.77.253:32861,127.27.77.252:42043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.252.keytab
--principal=kudu/127.27.77.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.252:42043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.77.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:33.836171 13470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:33.837170 13470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:33.845927 13480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:33.845971 13478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:33.846081 13470 server_base.cc:1034] running on GCE node
W20260806 00:56:33.846019 13477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:33.846776 13470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:33.847999 13470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:33.849218 13470 hybrid_clock.cc:648] HybridClock initialized: now 1785977793849204 us; error 32 us; skew 500 ppm
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:33.852131 13470 init.cc:377] Logged in from keytab as kudu/127.27.77.252@KRBTEST.COM (short username kudu)
I20260806 00:56:33.853040 13470 webserver.cc:460] Webserver started at http://127.27.77.252:39407/ using document root <none> and password file <none>
I20260806 00:56:33.853622 13470 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:33.853722 13470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:33.853961 13470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:56:33.856145 13470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/data/instance:
uuid: "4541b51d898040ae82cab9214a8b6207"
format_stamp: "Formatted at 2026-08-06 00:56:33 on dist-test-slave-sb2z"
server_key: "b5d2aa9d442fd74fa3de6706e37701bd"
server_key_iv: "cd45dd8e14761cd94bbce89629354cb2"
server_key_version: "encryptionkey@0"
I20260806 00:56:33.856747 13470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/wal/instance:
uuid: "4541b51d898040ae82cab9214a8b6207"
format_stamp: "Formatted at 2026-08-06 00:56:33 on dist-test-slave-sb2z"
server_key: "b5d2aa9d442fd74fa3de6706e37701bd"
server_key_iv: "cd45dd8e14761cd94bbce89629354cb2"
server_key_version: "encryptionkey@0"
I20260806 00:56:33.860565 13470 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:56:33.863003 13493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:33.864316 13470 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:56:33.864468 13470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/wal
uuid: "4541b51d898040ae82cab9214a8b6207"
format_stamp: "Formatted at 2026-08-06 00:56:33 on dist-test-slave-sb2z"
server_key: "b5d2aa9d442fd74fa3de6706e37701bd"
server_key_iv: "cd45dd8e14761cd94bbce89629354cb2"
server_key_version: "encryptionkey@0"
I20260806 00:56:33.864588 13470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:33.879693 13470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:33.882918 13470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:33.883172 13470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:33.890889 13470 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.252:42043
I20260806 00:56:33.890914 13554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.252:42043 every 8 connection(s)
I20260806 00:56:33.891971 13470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/data/info.pb
I20260806 00:56:33.894649 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 13470
I20260806 00:56:33.894795 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977619051187-27959-0/minicluster-data/master-2/wal/instance
I20260806 00:56:33.894816 13555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:33.895085 27959 external_mini_cluster.cc:1428] Setting key 9ff880b76e05fd6589f44d2cc95d2b97
I20260806 00:56:33.900182 13555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:56:33.902197 13372 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } attempt: 3
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.252@KRBTEST.COM
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for kudu/127.27.77.252@KRBTEST.COM
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.252@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:33.912051 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.896589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55786 (local address 127.27.77.252:42043)
0806 00:56:33.897031 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:33.897050 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:33.897091 (+ 41us) server_negotiation.cc:407] Connection header received
0806 00:56:33.897792 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:33.897814 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:33.898245 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:33.898624 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:33.899410 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.900972 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.902067 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.902915 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.903100 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.906223 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:33.906229 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:33.906247 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:33.906276 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:33.908483 (+ 2207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.909008 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.909014 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.909020 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.909187 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.909612 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.909616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.909619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.910175 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:33.910403 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:33.910684 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 00:56:33.910992 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":118,"threads_started":1}
I20260806 00:56:33.913129 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.902278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46226 (local address 127.27.77.254:45947)
0806 00:56:33.902397 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:33.902402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:33.902790 (+ 388us) server_negotiation.cc:407] Connection header received
0806 00:56:33.902995 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:33.902998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:33.903063 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:33.903116 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:33.904171 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.904990 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.906100 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.906479 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.906564 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.909273 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:33.909277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:33.909279 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:33.909304 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:33.910968 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.911452 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.911456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.911458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.911605 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.911934 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.911940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.911943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.912294 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:33.912396 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:33.912849 (+ 453us) server_negotiation.cc:299] Negotiation successful
0806 00:56:33.912985 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:56:33.913487 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.902615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42043 (local address 127.0.0.1:55790)
0806 00:56:33.902854 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.902866 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:33.902943 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:33.903416 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:33.903422 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:33.903657 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:33.903819 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.903829 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.904671 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.904675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.905386 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.905400 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.905997 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.906003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.906171 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.906658 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:33.906678 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:33.908197 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:33.911889 (+ 3692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.911894 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.911896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.912138 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.912462 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.912476 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.912478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.912570 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.913000 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:33.913003 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:33.913105 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 00:56:33.913217 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 00:56:33.913343 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":50}
I20260806 00:56:33.913766 13560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.902129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:45947 (local address 127.0.0.1:46226)
0806 00:56:33.902664 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.902716 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:33.902882 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:33.903307 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:33.903314 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:33.903828 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:33.904029 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.904037 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.905108 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.905111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.905964 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.905973 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.906738 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.906743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.906867 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.907375 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:33.907390 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:33.909141 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:33.911081 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.911090 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.911094 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.911344 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.911708 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.911712 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.911714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.911806 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.912483 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:33.912489 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:33.912615 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 00:56:33.913347 (+ 732us) client_negotiation.cc:240] Negotiation successful
0806 00:56:33.913563 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":116,"threads_started":1}
I20260806 00:56:33.914089 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.902879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55790 (local address 127.27.77.252:42043)
0806 00:56:33.903135 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:33.903144 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:33.903156 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:56:33.903185 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:33.903189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:33.903234 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:33.903378 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:33.903961 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.904553 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.905524 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.905864 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.905977 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.908374 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:33.908379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:33.908382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:33.908413 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:33.911779 (+ 3366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.912262 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.912266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.912267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.912352 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.912690 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.912693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.912694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.912885 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:33.912984 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:33.913826 (+ 842us) server_negotiation.cc:299] Negotiation successful
0806 00:56:33.913937 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":94,"threads_started":1}
I20260806 00:56:33.915339 13555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:56:33.917387 13372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8: Bootstrap starting.
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:33.920176 13372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:33.921195 13372 log.cc:824] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.252@KRBTEST.COM for kudu/127.27.77.253@KRBTEST.COM
I20260806 00:56:33.921691 13463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } attempt: 3
I20260806 00:56:33.924290 13372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8: No bootstrap required, opened a new log
I20260806 00:56:33.925284 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.914441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46232 (local address 127.27.77.254:45947)
0806 00:56:33.914544 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:33.914548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:33.914677 (+ 129us) server_negotiation.cc:407] Connection header received
0806 00:56:33.914793 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:33.914797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:33.914835 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:33.914906 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:33.915770 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.916400 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.917713 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.917938 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.918053 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.920650 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:33.920654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:33.920657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:33.920682 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:33.922403 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.923473 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.923476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.923477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.923603 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.923934 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.923937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.923939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.924146 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:33.924935 (+ 789us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:33.925018 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 00:56:33.925133 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 00:56:33.925810 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.916672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40738 (local address 127.27.77.253:32861)
0806 00:56:33.916829 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:33.916834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:33.916849 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:56:33.916892 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:33.916895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:33.916940 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:33.917029 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:33.917479 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.918180 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.919296 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.919650 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.919755 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.922234 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:33.922238 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:33.922240 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:33.922269 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:33.923938 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.924449 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.924452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.924454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.924540 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.924870 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.924873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.924875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.925090 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:33.925163 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:33.925534 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 00:56:33.925652 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 00:56:33.926388 13560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.915851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:32861 (local address 127.0.0.1:40738)
0806 00:56:33.915986 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.916000 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:33.916081 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:33.917070 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:33.917073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:33.917250 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:33.917337 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.917344 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.918308 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.918314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.919143 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.919158 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.919881 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.919885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.919990 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.920517 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:33.920531 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:33.922084 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:33.924079 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.924081 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.924083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.924323 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.924660 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.924663 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.924665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.924760 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.925239 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:33.925242 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:33.925330 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 00:56:33.926096 (+ 766us) client_negotiation.cc:240] Negotiation successful
0806 00:56:33.926237 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 00:56:33.927512 13555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.253@KRBTEST.COM for kudu/127.27.77.252@KRBTEST.COM
I20260806 00:56:33.929337 13372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } }
I20260806 00:56:33.929648 13372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:33.929746 13372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30bc0c578ec145f586991b480494d0b8, State: Initialized, Role: FOLLOWER
I20260806 00:56:33.930316 13372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } }
I20260806 00:56:33.931972 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.922104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42043 (local address 127.0.0.1:55798)
0806 00:56:33.922357 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.922373 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:33.922453 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:33.923029 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:33.923033 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:33.923230 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:33.923364 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.923372 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.924707 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.924712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.925552 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.925567 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.926196 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.926200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.926281 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.926719 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:33.926738 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:33.928269 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:33.930330 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.930333 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.930335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.930585 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.930903 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.930907 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.930909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.931008 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.931426 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:33.931429 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:33.931576 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 00:56:33.931673 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 00:56:33.931824 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":3}
I20260806 00:56:33.931962 13372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8 [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.252@KRBTEST.COM for kudu/127.27.77.252@KRBTEST.COM
I20260806 00:56:33.933699 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.922602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55798 (local address 127.27.77.252:42043)
0806 00:56:33.922760 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:33.922764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:33.922777 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:56:33.922814 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:33.922818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:33.922864 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:33.923743 (+ 879us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:33.923841 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.924569 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.925712 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.926059 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.926501 (+ 442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.928432 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:33.928438 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:33.928445 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:33.928474 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:33.930206 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.930708 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.930712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.930713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.930797 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.931121 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.931123 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.931125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.931314 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:33.932830 (+ 1516us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:33.932903 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 00:56:33.933494 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 00:56:33 dist-test-slave-sb2z krb5kdc[13268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.253@KRBTEST.COM
I20260806 00:56:33.937551 13463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b: Bootstrap starting.
I20260806 00:56:33.937906 13560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.928002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42043 (local address 127.0.0.1:55806)
0806 00:56:33.928432 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:33.928444 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:33.928522 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:33.928805 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:33.928807 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:33.928965 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:33.929039 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.929057 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.929976 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.929979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.930740 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:33.930752 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.931364 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.931367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:33.931435 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.931856 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:33.931870 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:33.933302 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:33.935237 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.935240 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.935243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.935469 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.936935 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:33.936938 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:33.936939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:33.937040 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:33.937456 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:33.937459 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:33.937530 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 00:56:33.937634 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 00:56:33.937761 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336}
I20260806 00:56:33.938345 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.928148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55806 (local address 127.27.77.252:42043)
0806 00:56:33.928321 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:33.928325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:33.928601 (+ 276us) server_negotiation.cc:407] Connection header received
0806 00:56:33.928626 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:33.928630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:33.928667 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:33.928724 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:33.929167 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.929863 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.930864 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.931176 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.931266 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.933455 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:33.933459 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:33.933461 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:33.933488 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:33.935140 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.935618 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.935621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.935622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.935710 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.937150 (+ 1440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.937153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.937154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.937342 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:33.937425 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:33.938105 (+ 680us) server_negotiation.cc:299] Negotiation successful
0806 00:56:33.938221 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 00:56:33.938982 13566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } } }
I20260806 00:56:33.939107 13566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 00:56:33.939332 13582 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:56:33.939417 13582 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:56:33.940624 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.927151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40750 (local address 127.27.77.253:32861)
0806 00:56:33.927261 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:33.927265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:33.927841 (+ 576us) server_negotiation.cc:407] Connection header received
0806 00:56:33.927879 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:33.927881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:33.927927 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:33.928001 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:33.929587 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.930337 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.931426 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.932580 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.932693 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.936573 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:33.936577 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:33.936579 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:33.936605 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:33.938234 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.938750 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.938754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.938759 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.938853 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.939164 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.939167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.939169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.939401 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:33.939607 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:33.940299 (+ 692us) server_negotiation.cc:299] Negotiation successful
0806 00:56:33.940441 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":795,"server-negotiator.queue_time_us":49}
I20260806 00:56:33.942589 13463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:33.943197 13555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207: Bootstrap starting.
I20260806 00:56:33.943451 13463 log.cc:824] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:33.939702 13285 master_runner.cc:186] Error getting master registration for 127.27.77.253:32861: OK, instance_id { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" instance_seqno: 1785977793666630 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 00:56:33.944193 13285 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:56:33.945405 13463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b: No bootstrap required, opened a new log
I20260806 00:56:33.945623 13555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:33.946332 13555 log.cc:824] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:33.948451 13555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207: No bootstrap required, opened a new log
I20260806 00:56:33.949167 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:56:33.949226 13463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } }
I20260806 00:56:33.949457 13463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:33.949515 13463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 769d0f1c6728420d9e92e3f35c8a198b, State: Initialized, Role: FOLLOWER
I20260806 00:56:33.950057 13463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } }
I20260806 00:56:33.951452 13463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:56:33.952548 13555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } }
I20260806 00:56:33.952821 13555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:33.952888 13555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4541b51d898040ae82cab9214a8b6207, State: Initialized, Role: FOLLOWER
I20260806 00:56:33.953387 13555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } }
I20260806 00:56:33.954715 13555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 00:56:33.956974 13607 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:56:33.957113 13607 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:56:33.959846 13588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } } }
I20260806 00:56:33.960008 13588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [sys.catalog]: This master's current role is: FOLLOWER
W20260806 00:56:33.961400 13625 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 00:56:33.961583 13381 master_runner.cc:263] No leader master found from master 769d0f1c6728420d9e92e3f35c8a198b
I20260806 00:56:33.961656 13381 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 00:56:33.961728 13625 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:56:33.957466 13608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } } }
I20260806 00:56:33.961985 13608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:56:33.965716 13470 master_runner.cc:263] No leader master found from master 4541b51d898040ae82cab9214a8b6207
I20260806 00:56:33.965801 13470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:56:33.967125 13588 raft_consensus.cc:482] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:56:33.967317 13588 raft_consensus.cc:504] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } }
I20260806 00:56:33.969002 13343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "769d0f1c6728420d9e92e3f35c8a198b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30bc0c578ec145f586991b480494d0b8" is_pre_election: true
I20260806 00:56:33.969357 13343 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 769d0f1c6728420d9e92e3f35c8a198b in term 0.
I20260806 00:56:33.969404 13588 leader_election.cc:288] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30bc0c578ec145f586991b480494d0b8 (127.27.77.254:45947), 4541b51d898040ae82cab9214a8b6207 (127.27.77.252:42043)
I20260806 00:56:33.969563 13524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "769d0f1c6728420d9e92e3f35c8a198b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4541b51d898040ae82cab9214a8b6207" is_pre_election: true
I20260806 00:56:33.969859 13524 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 769d0f1c6728420d9e92e3f35c8a198b in term 0.
I20260806 00:56:33.969832 13401 leader_election.cc:302] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30bc0c578ec145f586991b480494d0b8, 769d0f1c6728420d9e92e3f35c8a198b; no voters:
I20260806 00:56:33.970803 13588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:56:33.970901 13588 raft_consensus.cc:482] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:56:33.970955 13588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:33.973287 13588 raft_consensus.cc:504] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } }
I20260806 00:56:33.973675 13588 leader_election.cc:288] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [CANDIDATE]: Term 1 election: Requested vote from peers 30bc0c578ec145f586991b480494d0b8 (127.27.77.254:45947), 4541b51d898040ae82cab9214a8b6207 (127.27.77.252:42043)
I20260806 00:56:33.974222 13524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "769d0f1c6728420d9e92e3f35c8a198b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4541b51d898040ae82cab9214a8b6207"
I20260806 00:56:33.974344 13524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:33.976686 13524 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 769d0f1c6728420d9e92e3f35c8a198b in term 1.
I20260806 00:56:33.977082 13402 leader_election.cc:302] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4541b51d898040ae82cab9214a8b6207, 769d0f1c6728420d9e92e3f35c8a198b; no voters:
I20260806 00:56:33.977415 13343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "769d0f1c6728420d9e92e3f35c8a198b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30bc0c578ec145f586991b480494d0b8"
I20260806 00:56:33.977574 13343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:33.978669 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.958921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55812 (local address 127.27.77.252:42043)
0806 00:56:33.959117 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:33.959122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:33.966784 (+ 7662us) server_negotiation.cc:407] Connection header received
0806 00:56:33.966948 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:33.966951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:33.967009 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:33.967060 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:33.968163 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.968921 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.970926 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.971293 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.971376 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.974622 (+ 3246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:33.974626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:33.974631 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:33.974663 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:33.976497 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.977087 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.977092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.977095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.977191 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.977559 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.977563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.977565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.977819 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:33.977943 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:33.978340 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 00:56:33.978475 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260806 00:56:33.978950 13343 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 769d0f1c6728420d9e92e3f35c8a198b in term 1.
I20260806 00:56:33.979707 13588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:56:33.979974 13588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [term 1 LEADER]: Becoming Leader. State: Replica: 769d0f1c6728420d9e92e3f35c8a198b, State: Running, Role: LEADER
I20260806 00:56:33.980334 13588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } }
I20260806 00:56:33.982215 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.960230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40752 (local address 127.27.77.253:32861)
0806 00:56:33.960376 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:33.960380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:33.962001 (+ 1621us) server_negotiation.cc:407] Connection header received
0806 00:56:33.962220 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:33.962224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:33.962274 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:33.962326 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:33.963408 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.964308 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.966274 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.966690 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.966845 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.971722 (+ 4877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:33.971726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:33.971729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:33.971765 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:33.973632 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.980641 (+ 7009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.980646 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.980648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.980759 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.981127 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.981131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.981133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.981375 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:33.981528 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:33.981910 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 00:56:33.982045 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 00:56:33.982420 13588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 769d0f1c6728420d9e92e3f35c8a198b. Latest consensus state: current_term: 1 leader_uuid: "769d0f1c6728420d9e92e3f35c8a198b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } } }
I20260806 00:56:33.982638 13588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:33.990212 13633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:56:33.991357 13633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:56:33.998453 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:33.960100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46242 (local address 127.27.77.254:45947)
0806 00:56:33.960281 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:33.960286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:33.966451 (+ 6165us) server_negotiation.cc:407] Connection header received
0806 00:56:33.966580 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:33.966583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:33.966639 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:33.966730 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:33.982953 (+ 16223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.983884 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.988873 (+ 4989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:33.989285 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:33.989375 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:33.992827 (+ 3452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:33.992831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:33.992835 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:33.992869 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:33.994689 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.995237 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.995241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.995243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.995349 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:33.995745 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:33.995749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:33.995751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:33.995995 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:33.996132 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:33.998122 (+ 1990us) server_negotiation.cc:299] Negotiation successful
0806 00:56:33.998270 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 00:56:34.005956 13524 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207 [term 1 FOLLOWER]: Refusing update from remote peer 769d0f1c6728420d9e92e3f35c8a198b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:56:34.006383 13343 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8 [term 1 FOLLOWER]: Refusing update from remote peer 769d0f1c6728420d9e92e3f35c8a198b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:56:34.007441 13631 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:56:34.008064 13588 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [LEADER]: Connected to new peer: Peer: permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:56:34.016904 13608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 769d0f1c6728420d9e92e3f35c8a198b. Latest consensus state: current_term: 1 leader_uuid: "769d0f1c6728420d9e92e3f35c8a198b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } } }
I20260806 00:56:34.017047 13608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:56:34.018159 13566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 769d0f1c6728420d9e92e3f35c8a198b. Latest consensus state: current_term: 1 leader_uuid: "769d0f1c6728420d9e92e3f35c8a198b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } } }
I20260806 00:56:34.018286 13566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:56:34.028745 13588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "769d0f1c6728420d9e92e3f35c8a198b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } } }
I20260806 00:56:34.028898 13588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:34.029016 13635 mvcc.cc:205] Tried to move back new op lower bound from 7315365044239003648 to 7315365044147556352. Current Snapshot: MvccSnapshot[applied={T|T < 7315365044239003648}]
I20260806 00:56:34.030544 13608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "769d0f1c6728420d9e92e3f35c8a198b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } } }
I20260806 00:56:34.030689 13608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:56:34.032992 13637 mvcc.cc:205] Tried to move back new op lower bound from 7315365044239003648 to 7315365044147556352. Current Snapshot: MvccSnapshot[applied={T|T < 7315365044239003648}]
I20260806 00:56:34.034250 13631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "769d0f1c6728420d9e92e3f35c8a198b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } } }
I20260806 00:56:34.034386 13631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:34.037662 13566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "769d0f1c6728420d9e92e3f35c8a198b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30bc0c578ec145f586991b480494d0b8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 45947 } } peers { permanent_uuid: "769d0f1c6728420d9e92e3f35c8a198b" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 32861 } } peers { permanent_uuid: "4541b51d898040ae82cab9214a8b6207" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42043 } } }
I20260806 00:56:34.037779 13566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:56:34.038782 13636 mvcc.cc:205] Tried to move back new op lower bound from 7315365044239003648 to 7315365044147556352. Current Snapshot: MvccSnapshot[applied={T|T < 7315365044239003648}]
I20260806 00:56:34.043676 13633 catalog_manager.cc:1347] Generated new cluster ID: ba7c107df166461ca6f9dfae75c5ca03
I20260806 00:56:34.043748 13633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:56:34.075776 13633 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:56:34.077314 13633 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:56:34.100332 13633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 769d0f1c6728420d9e92e3f35c8a198b: Generated new TSK 0
I20260806 00:56:34.101308 13633 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:56:34.940847 13582 catalog_manager.cc:1259] Loaded cluster ID: ba7c107df166461ca6f9dfae75c5ca03
I20260806 00:56:34.940961 13582 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8: loading cluster ID for follower catalog manager: success
I20260806 00:56:34.942714 13582 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8: acquiring CA information for follower catalog manager: success
I20260806 00:56:34.943956 13582 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 30bc0c578ec145f586991b480494d0b8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:56:34.963068 13625 catalog_manager.cc:1259] Loaded cluster ID: ba7c107df166461ca6f9dfae75c5ca03
I20260806 00:56:34.963148 13625 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207: loading cluster ID for follower catalog manager: success
I20260806 00:56:34.964908 13625 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207: acquiring CA information for follower catalog manager: success
I20260806 00:56:34.966107 13625 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4541b51d898040ae82cab9214a8b6207: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:56:35.174263 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 13285
I20260806 00:56:35.186182 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 13381
I20260806 00:56:35.196087 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 13470
2026-08-06T00:56:35Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3320 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:56:35 dist-test-slave-sb2z krb5kdc[13652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:56:35 dist-test-slave-sb2z krb5kdc[13652](info): set up 2 sockets
Aug 06 00:56:35 dist-test-slave-sb2z krb5kdc[13652](info): commencing operation
krb5kdc: starting...
W20260806 00:56:36.445518 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:56:36 dist-test-slave-sb2z krb5kdc[13652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:56:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:56:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:56:36.582520 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:40243
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:40699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:40243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:36.697326 13670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:36.697687 13670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:36.697774 13670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:36.701896 13670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:56:36.702009 13670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:36.702051 13670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:36.702104 13670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:56:36.702157 13670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:56:36.707556 13670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:40699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:40243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:40243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:36.708683 13670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:36.709755 13670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:36.719108 13678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:36.719130 13679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:36.719108 13681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:36.719486 13670 server_base.cc:1034] running on GCE node
I20260806 00:56:36.720178 13670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:36.721385 13670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:36.722605 13670 hybrid_clock.cc:648] HybridClock initialized: now 1785977796722591 us; error 39 us; skew 500 ppm
Aug 06 00:56:36 dist-test-slave-sb2z krb5kdc[13652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977796, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:36.725492 13670 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:56:36.726364 13670 webserver.cc:460] Webserver started at http://127.27.77.254:45261/ using document root <none> and password file <none>
I20260806 00:56:36.726944 13670 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:36.727039 13670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:36.727286 13670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:56:36.729455 13670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "929befac94a54df2b7fd200531894a3c"
format_stamp: "Formatted at 2026-08-06 00:56:36 on dist-test-slave-sb2z"
server_key: "d2234003d4a8d00f8baa87e7f951c738"
server_key_iv: "025824358f376556d68d90ce92f3f987"
server_key_version: "encryptionkey@0"
I20260806 00:56:36.730041 13670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "929befac94a54df2b7fd200531894a3c"
format_stamp: "Formatted at 2026-08-06 00:56:36 on dist-test-slave-sb2z"
server_key: "d2234003d4a8d00f8baa87e7f951c738"
server_key_iv: "025824358f376556d68d90ce92f3f987"
server_key_version: "encryptionkey@0"
I20260806 00:56:36.733834 13670 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:56:36.736354 13692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:36.737614 13670 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:56:36.737762 13670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "929befac94a54df2b7fd200531894a3c"
format_stamp: "Formatted at 2026-08-06 00:56:36 on dist-test-slave-sb2z"
server_key: "d2234003d4a8d00f8baa87e7f951c738"
server_key_iv: "025824358f376556d68d90ce92f3f987"
server_key_version: "encryptionkey@0"
I20260806 00:56:36.737881 13670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:36.756747 13670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:36.760347 13670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:36.760604 13670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:36.769342 13670 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:40243
I20260806 00:56:36.769371 13764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:40243 every 8 connection(s)
I20260806 00:56:36.770460 13670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:56:36.773633 13765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:36.779804 13765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c: Bootstrap starting.
I20260806 00:56:36.780645 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 13670
I20260806 00:56:36.780781 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:56:36.781082 27959 external_mini_cluster.cc:1428] Setting key f8096a29fe82fa25a180adcdd37bed12
I20260806 00:56:36.782969 13765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:36.783979 13765 log.cc:824] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:36.786412 13765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c: No bootstrap required, opened a new log
Aug 06 00:56:36 dist-test-slave-sb2z krb5kdc[13652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:36.791203 13765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929befac94a54df2b7fd200531894a3c" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40243 } }
I20260806 00:56:36.791530 13765 raft_consensus.cc:374] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:36.791625 13765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929befac94a54df2b7fd200531894a3c, State: Initialized, Role: FOLLOWER
I20260806 00:56:36.792172 13765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929befac94a54df2b7fd200531894a3c" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40243 } }
I20260806 00:56:36.792369 13765 raft_consensus.cc:388] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:56:36.792464 13765 raft_consensus.cc:482] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:56:36.792614 13765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:36.793577 13765 raft_consensus.cc:504] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929befac94a54df2b7fd200531894a3c" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40243 } }
I20260806 00:56:36.793962 13765 leader_election.cc:302] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 929befac94a54df2b7fd200531894a3c; no voters:
I20260806 00:56:36.794330 13765 leader_election.cc:288] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:56:36.794405 13770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:56:36.794668 13770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [term 1 LEADER]: Becoming Leader. State: Replica: 929befac94a54df2b7fd200531894a3c, State: Running, Role: LEADER
I20260806 00:56:36.795047 13770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929befac94a54df2b7fd200531894a3c" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40243 } }
I20260806 00:56:36.795796 13765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:56:36.797288 13772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 929befac94a54df2b7fd200531894a3c. Latest consensus state: current_term: 1 leader_uuid: "929befac94a54df2b7fd200531894a3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929befac94a54df2b7fd200531894a3c" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40243 } } }
I20260806 00:56:36.797326 13771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "929befac94a54df2b7fd200531894a3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929befac94a54df2b7fd200531894a3c" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40243 } } }
I20260806 00:56:36.797493 13772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:36.797505 13771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:36.797677 13768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:36.782479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46328 (local address 127.27.77.254:40243)
0806 00:56:36.782982 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:36.783006 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:36.783056 (+ 50us) server_negotiation.cc:407] Connection header received
0806 00:56:36.784085 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:36.784112 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:36.784553 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:36.784931 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:36.785618 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:36.787212 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:36.788144 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:36.788927 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:36.789106 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:36.792039 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:36.792045 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:36.792062 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:36.792096 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:36.794133 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:36.794731 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:36.794737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:36.794743 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:36.794884 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:36.795195 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:36.795198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:36.795202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:36.795704 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:36.795908 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:36.796258 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 00:56:36.796590 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":134,"threads_started":1}
I20260806 00:56:36.801141 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 00:56:36.801173 13789 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:56:36.801295 13789 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:56:36.801378 13783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:56:36.802130 13783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:56:36.808271 13783 catalog_manager.cc:1347] Generated new cluster ID: 2aa1cd27d3084d1daa3e7f1bab3981e9
I20260806 00:56:36.808341 13783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:56:36.818816 13783 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:56:36.820101 13783 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:56:36.830464 13783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 929befac94a54df2b7fd200531894a3c: Generated new TSK 0
I20260806 00:56:36.831283 13783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:56:36.858012 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 13670
2026-08-06T00:56:36Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1661 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:56:36 dist-test-slave-sb2z krb5kdc[13799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:56:36 dist-test-slave-sb2z krb5kdc[13799](info): set up 2 sockets
Aug 06 00:56:36 dist-test-slave-sb2z krb5kdc[13799](info): commencing operation
krb5kdc: starting...
W20260806 00:56:38.096665 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:56:38 dist-test-slave-sb2z krb5kdc[13799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:56:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:56:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:56:38.231253 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:41297
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:39101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:41297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:38.346319 13818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:38.346597 13818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:38.346640 13818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:38.350828 13818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:56:38.350893 13818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:38.350914 13818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:38.350931 13818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:56:38.350993 13818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:56:38.356494 13818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:39101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:41297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:41297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:38.357656 13818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:38.358733 13818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:38.367444 13827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:38.367583 13831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:38.367491 13825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:38.367918 13818 server_base.cc:1034] running on GCE node
I20260806 00:56:38.368501 13818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:38.369612 13818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:38.370802 13818 hybrid_clock.cc:648] HybridClock initialized: now 1785977798370745 us; error 73 us; skew 500 ppm
Aug 06 00:56:38 dist-test-slave-sb2z krb5kdc[13799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977798, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:38.377935 13818 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:56:38.378939 13818 webserver.cc:460] Webserver started at http://127.27.77.254:42483/ using document root <none> and password file <none>
I20260806 00:56:38.379472 13818 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:38.379603 13818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:38.379805 13818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:56:38.381846 13818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "74621d8113284ab7abe2a8be5ab6ca41"
format_stamp: "Formatted at 2026-08-06 00:56:38 on dist-test-slave-sb2z"
server_key: "94af1c229611d3d503461233998305a7"
server_key_iv: "ff7fb31f8ca99ba1f402d23284928e3c"
server_key_version: "encryptionkey@0"
I20260806 00:56:38.382372 13818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "74621d8113284ab7abe2a8be5ab6ca41"
format_stamp: "Formatted at 2026-08-06 00:56:38 on dist-test-slave-sb2z"
server_key: "94af1c229611d3d503461233998305a7"
server_key_iv: "ff7fb31f8ca99ba1f402d23284928e3c"
server_key_version: "encryptionkey@0"
I20260806 00:56:38.386227 13818 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:56:38.388775 13838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:38.390103 13818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:56:38.390249 13818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "74621d8113284ab7abe2a8be5ab6ca41"
format_stamp: "Formatted at 2026-08-06 00:56:38 on dist-test-slave-sb2z"
server_key: "94af1c229611d3d503461233998305a7"
server_key_iv: "ff7fb31f8ca99ba1f402d23284928e3c"
server_key_version: "encryptionkey@0"
I20260806 00:56:38.390363 13818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:38.422801 13818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:38.426245 13818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:38.426467 13818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:38.434172 13818 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:41297
I20260806 00:56:38.434505 13907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:41297 every 8 connection(s)
I20260806 00:56:38.435226 13818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:56:38.438303 13908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:38.439124 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 13818
I20260806 00:56:38.439267 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:56:38.439607 27959 external_mini_cluster.cc:1428] Setting key be853608bc3bf9ff296c3819b3a92f8d
I20260806 00:56:38.445235 13908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: Bootstrap starting.
I20260806 00:56:38.447933 13908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:38.448678 13908 log.cc:824] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:56:38 dist-test-slave-sb2z krb5kdc[13799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:38.451073 13908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: No bootstrap required, opened a new log
I20260806 00:56:38.455205 13913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:38.441085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36172 (local address 127.27.77.254:41297)
0806 00:56:38.441527 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:38.441552 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:38.441599 (+ 47us) server_negotiation.cc:407] Connection header received
0806 00:56:38.442418 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:38.442444 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:38.442870 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:38.443248 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:38.444231 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:38.445652 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:38.446533 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:38.447190 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:38.447352 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:38.450073 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:38.450079 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:38.450095 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:38.450126 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:38.452049 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:38.452478 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:38.452484 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:38.452490 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:38.452629 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:38.452888 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:38.452891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:38.452893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:38.453465 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:38.453669 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:38.453922 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 00:56:38.454227 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":105,"threads_started":1}
I20260806 00:56:38.455742 13908 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } }
I20260806 00:56:38.455992 13908 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:38.456076 13908 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74621d8113284ab7abe2a8be5ab6ca41, State: Initialized, Role: FOLLOWER
I20260806 00:56:38.456547 13908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } }
I20260806 00:56:38.456702 13908 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:56:38.456768 13908 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:56:38.456864 13908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:38.458012 13908 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } }
I20260806 00:56:38.458302 13908 leader_election.cc:302] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74621d8113284ab7abe2a8be5ab6ca41; no voters:
I20260806 00:56:38.458554 13908 leader_election.cc:288] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:56:38.458701 13917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:56:38.458961 13917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 1 LEADER]: Becoming Leader. State: Replica: 74621d8113284ab7abe2a8be5ab6ca41, State: Running, Role: LEADER
I20260806 00:56:38.459342 13917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } }
I20260806 00:56:38.459882 13908 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:56:38.462412 13918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74621d8113284ab7abe2a8be5ab6ca41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } } }
I20260806 00:56:38.462551 13918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:38.462852 13919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74621d8113284ab7abe2a8be5ab6ca41. Latest consensus state: current_term: 1 leader_uuid: "74621d8113284ab7abe2a8be5ab6ca41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } } }
I20260806 00:56:38.463150 13919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [sys.catalog]: This master's current role is: LEADER
W20260806 00:56:38.465243 13939 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:56:38.465313 13939 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:56:38.465395 13937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:56:38.466091 13937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:56:38.471410 13937 catalog_manager.cc:1347] Generated new cluster ID: ff25458b51a84697a11be8b4b137840f
I20260806 00:56:38.471479 13937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:56:38.480399 13937 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:56:38.481724 13937 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:56:38.490240 13937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: Generated new TSK 0
I20260806 00:56:38.491088 13937 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:56:38.530778 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:41297
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:39101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:38.650293 13946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:38.650569 13946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:38.650614 13946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:38.654680 13946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:38.654744 13946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:38.654804 13946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:56:38.660214 13946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:39101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:38.661514 13946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:38.662631 13946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:38.672495 13951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:38.672505 13953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:38.672505 13955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:38.672729 13946 server_base.cc:1034] running on GCE node
I20260806 00:56:38.673210 13946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:38.673827 13946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:38.675076 13946 hybrid_clock.cc:648] HybridClock initialized: now 1785977798675055 us; error 32 us; skew 500 ppm
Aug 06 00:56:38 dist-test-slave-sb2z krb5kdc[13799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977798, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:38.678248 13946 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:56:38.679175 13946 webserver.cc:460] Webserver started at http://127.27.77.193:35169/ using document root <none> and password file <none>
I20260806 00:56:38.679764 13946 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:38.679827 13946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:38.679998 13946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:56:38.681982 13946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "90a5681b349d4313a8a0ea9bec4d87fd"
format_stamp: "Formatted at 2026-08-06 00:56:38 on dist-test-slave-sb2z"
server_key: "34ed7a9f34fac7e3a891f068f5c0771c"
server_key_iv: "7a22f727c9686fa7af94a57ea5720d6e"
server_key_version: "encryptionkey@0"
I20260806 00:56:38.682515 13946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "90a5681b349d4313a8a0ea9bec4d87fd"
format_stamp: "Formatted at 2026-08-06 00:56:38 on dist-test-slave-sb2z"
server_key: "34ed7a9f34fac7e3a891f068f5c0771c"
server_key_iv: "7a22f727c9686fa7af94a57ea5720d6e"
server_key_version: "encryptionkey@0"
I20260806 00:56:38.686553 13946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 00:56:38.689016 13965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:38.690291 13946 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:56:38.690421 13946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "90a5681b349d4313a8a0ea9bec4d87fd"
format_stamp: "Formatted at 2026-08-06 00:56:38 on dist-test-slave-sb2z"
server_key: "34ed7a9f34fac7e3a891f068f5c0771c"
server_key_iv: "7a22f727c9686fa7af94a57ea5720d6e"
server_key_version: "encryptionkey@0"
I20260806 00:56:38.690538 13946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:38.704360 13946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:38.707811 13946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:38.708035 13946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:38.708673 13946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:38.709551 13946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:38.709601 13946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:38.709673 13946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:38.709703 13946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:38.719220 13946 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:35109
I20260806 00:56:38.719276 14114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:35109 every 8 connection(s)
I20260806 00:56:38.720274 13946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:56:38.729084 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 13946
I20260806 00:56:38.729269 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:56:38.729561 27959 external_mini_cluster.cc:1428] Setting key 1ec750b51ed0edc982bbda42dfea5d36
Aug 06 00:56:38 dist-test-slave-sb2z krb5kdc[13799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977798, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:38.735260 13913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:38.722200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:54373 (local address 127.27.77.254:41297)
0806 00:56:38.722347 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:38.722351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:38.723216 (+ 865us) server_negotiation.cc:407] Connection header received
0806 00:56:38.724713 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:38.724717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:38.724775 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:38.724886 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:38.726068 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:38.726620 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:38.727855 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:38.728101 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:38.728218 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:38.731050 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:38.731053 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:38.731055 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:38.731078 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:38.732947 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:38.733523 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:38.733529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:38.733533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:38.733650 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:38.734094 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:38.734096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:38.734097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:38.734273 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:38.734349 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:38.735022 (+ 673us) server_negotiation.cc:299] Negotiation successful
0806 00:56:38.735134 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:56:38.736554 14118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:38.722577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:54373)
0806 00:56:38.723051 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:38.723101 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:38.724474 (+ 1373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:38.725063 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:38.725069 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:38.725424 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:38.725903 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:38.725917 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:38.726764 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:38.726767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:38.727723 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:38.727733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:38.728267 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:38.728272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:38.728439 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:38.729102 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:38.729119 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:38.730842 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:38.733098 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:38.733103 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:38.733115 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:38.733389 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:38.733767 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:38.733772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:38.733776 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:38.733980 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:38.734381 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:38.734392 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:38.734742 (+ 350us) client_negotiation.cc:769] Sending connection context
0806 00:56:38.734988 (+ 246us) client_negotiation.cc:240] Negotiation successful
0806 00:56:38.735381 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":117,"threads_started":1}
I20260806 00:56:38.737978 14115 heartbeater.cc:347] Connected to a master server at 127.27.77.254:41297
I20260806 00:56:38.738234 14115 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:38.738746 14115 heartbeater.cc:511] Master 127.27.77.254:41297 requested a full tablet report, sending...
I20260806 00:56:38.740311 13861 ts_manager.cc:194] Registered new tserver with Master: 90a5681b349d4313a8a0ea9bec4d87fd (127.27.77.193:35109)
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
I20260806 00:56:38.741513 13861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:54373
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:56:38.762641 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:41297
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:39101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:38.878445 14122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:38.878751 14122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:38.878840 14122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:38.882997 14122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:38.883118 14122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:38.883226 14122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:56:38.888675 14122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:39101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:38.889822 14122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:38.890863 14122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:38.900894 14130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:38.900939 14129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:38.901077 14122 server_base.cc:1034] running on GCE node
W20260806 00:56:38.901027 14132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:38.901731 14122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:38.902380 14122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:38.903613 14122 hybrid_clock.cc:648] HybridClock initialized: now 1785977798903586 us; error 40 us; skew 500 ppm
Aug 06 00:56:38 dist-test-slave-sb2z krb5kdc[13799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977798, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:38.906417 14122 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:56:38.907321 14122 webserver.cc:460] Webserver started at http://127.27.77.194:45961/ using document root <none> and password file <none>
I20260806 00:56:38.907920 14122 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:38.907982 14122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:38.908154 14122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:56:38.910162 14122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "ace5781cc9314ccf8d24279b51e2cf68"
format_stamp: "Formatted at 2026-08-06 00:56:38 on dist-test-slave-sb2z"
server_key: "8322a617b3a5648aec8b13549e6c80a0"
server_key_iv: "fa84f641a1e29ccecd6965f17e0a7c5a"
server_key_version: "encryptionkey@0"
I20260806 00:56:38.910686 14122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "ace5781cc9314ccf8d24279b51e2cf68"
format_stamp: "Formatted at 2026-08-06 00:56:38 on dist-test-slave-sb2z"
server_key: "8322a617b3a5648aec8b13549e6c80a0"
server_key_iv: "fa84f641a1e29ccecd6965f17e0a7c5a"
server_key_version: "encryptionkey@0"
I20260806 00:56:38.914363 14122 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 00:56:38.916872 14141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:38.918227 14122 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:56:38.918377 14122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "ace5781cc9314ccf8d24279b51e2cf68"
format_stamp: "Formatted at 2026-08-06 00:56:38 on dist-test-slave-sb2z"
server_key: "8322a617b3a5648aec8b13549e6c80a0"
server_key_iv: "fa84f641a1e29ccecd6965f17e0a7c5a"
server_key_version: "encryptionkey@0"
I20260806 00:56:38.918497 14122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:38.943634 14122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:38.947270 14122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:38.947502 14122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:38.948228 14122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:38.949115 14122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:38.949173 14122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:38.949214 14122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:38.949268 14122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:38.959961 14122 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:36549
I20260806 00:56:38.959978 14295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:36549 every 8 connection(s)
I20260806 00:56:38.961174 14122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:56:38.970758 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 14122
I20260806 00:56:38.970896 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:56:38.971223 27959 external_mini_cluster.cc:1428] Setting key a9088c3d998f4ea0c6a1397eb446aa8a
Aug 06 00:56:38 dist-test-slave-sb2z krb5kdc[13799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977798, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:38.977221 13913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:38.963356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:46833 (local address 127.27.77.254:41297)
0806 00:56:38.963491 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:38.963495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:38.964494 (+ 999us) server_negotiation.cc:407] Connection header received
0806 00:56:38.966078 (+ 1584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:38.966081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:38.966152 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:38.966259 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:38.967456 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:38.968057 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:38.969259 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:38.969494 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:38.969613 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:38.972796 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:38.972800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:38.972802 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:38.972834 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:38.974345 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:38.975064 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:38.975070 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:38.975073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:38.975246 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:38.975870 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:38.975874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:38.975875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:38.976087 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:38.976216 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:38.976944 (+ 728us) server_negotiation.cc:299] Negotiation successful
0806 00:56:38.977061 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:56:38.978278 14299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:38.963852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:46833)
0806 00:56:38.964343 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:38.964386 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:38.965720 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:38.966440 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:38.966446 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:38.966825 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:38.967301 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:38.967314 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:38.968211 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:38.968214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:38.969112 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:38.969122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:38.969667 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:38.969671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:38.969839 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:38.970436 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:38.970479 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:38.972611 (+ 2132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:38.974477 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:38.974488 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:38.974509 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:38.974952 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:38.975378 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:38.975385 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:38.975389 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:38.975669 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:38.976245 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:38.976257 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:38.976641 (+ 384us) client_negotiation.cc:769] Sending connection context
0806 00:56:38.976889 (+ 248us) client_negotiation.cc:240] Negotiation successful
0806 00:56:38.977328 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":116,"threads_started":1}
I20260806 00:56:38.979801 14297 heartbeater.cc:347] Connected to a master server at 127.27.77.254:41297
I20260806 00:56:38.980072 14297 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:38.980587 14297 heartbeater.cc:511] Master 127.27.77.254:41297 requested a full tablet report, sending...
I20260806 00:56:38.981802 13861 ts_manager.cc:194] Registered new tserver with Master: ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549)
I20260806 00:56:38.982669 13861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:46833
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:56:39.006742 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:41297
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:39101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:39.125219 14303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:39.125502 14303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:39.125583 14303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:39.129714 14303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:39.129781 14303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:39.129871 14303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:56:39.135246 14303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:39101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:39.136420 14303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:39.137511 14303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:39.147697 14313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:39.147689 14309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:39.147667 14310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:39.148048 14303 server_base.cc:1034] running on GCE node
I20260806 00:56:39.148541 14303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:39.149240 14303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:39.150470 14303 hybrid_clock.cc:648] HybridClock initialized: now 1785977799150457 us; error 27 us; skew 500 ppm
Aug 06 00:56:39 dist-test-slave-sb2z krb5kdc[13799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977799, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:39.153574 14303 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:56:39.154518 14303 webserver.cc:460] Webserver started at http://127.27.77.195:43673/ using document root <none> and password file <none>
I20260806 00:56:39.155112 14303 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:39.155207 14303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:39.155447 14303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:56:39.157573 14303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "0ab771c8e62b4bfd83206f3ee3166761"
format_stamp: "Formatted at 2026-08-06 00:56:39 on dist-test-slave-sb2z"
server_key: "cf482e91ee8011546b9132c1649717e3"
server_key_iv: "faea17daf0cabeaa4c737ee384392108"
server_key_version: "encryptionkey@0"
I20260806 00:56:39.158162 14303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "0ab771c8e62b4bfd83206f3ee3166761"
format_stamp: "Formatted at 2026-08-06 00:56:39 on dist-test-slave-sb2z"
server_key: "cf482e91ee8011546b9132c1649717e3"
server_key_iv: "faea17daf0cabeaa4c737ee384392108"
server_key_version: "encryptionkey@0"
I20260806 00:56:39.162387 14303 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 00:56:39.164983 14321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:39.166273 14303 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:56:39.166425 14303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "0ab771c8e62b4bfd83206f3ee3166761"
format_stamp: "Formatted at 2026-08-06 00:56:39 on dist-test-slave-sb2z"
server_key: "cf482e91ee8011546b9132c1649717e3"
server_key_iv: "faea17daf0cabeaa4c737ee384392108"
server_key_version: "encryptionkey@0"
I20260806 00:56:39.166544 14303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:39.185024 14303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:39.188359 14303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:39.188576 14303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:39.189219 14303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:39.190075 14303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:39.190148 14303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:39.190220 14303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:39.190271 14303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:39.200325 14303 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:44175
I20260806 00:56:39.200354 14487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:44175 every 8 connection(s)
I20260806 00:56:39.201293 14303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:56:39.204826 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 14303
I20260806 00:56:39.204957 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:56:39.205276 27959 external_mini_cluster.cc:1428] Setting key e56204bbc4aa3b7e41bb18eb4ebd3dc9
Aug 06 00:56:39 dist-test-slave-sb2z krb5kdc[13799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977799, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:39.216746 13913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.203564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:41511 (local address 127.27.77.254:41297)
0806 00:56:39.203713 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:39.203718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:39.204759 (+ 1041us) server_negotiation.cc:407] Connection header received
0806 00:56:39.206627 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:39.206630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:39.206682 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:39.206759 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:39.208385 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.208937 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.210115 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.210346 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.210438 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.213207 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:39.213210 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:39.213212 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:39.213239 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:39.214730 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:39.215249 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:39.215252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:39.215253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:39.215329 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:39.215738 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:39.215740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:39.215741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:39.215925 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:39.216001 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:39.216508 (+ 507us) server_negotiation.cc:299] Negotiation successful
0806 00:56:39.216614 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:56:39.217682 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.203980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:41511)
0806 00:56:39.204533 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:39.204600 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:39.206331 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:39.207011 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:39.207021 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:39.207458 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:39.208220 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.208236 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.209060 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.209063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.209971 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.209982 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.210500 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.210503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.210649 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.211394 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:39.211412 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:39.213036 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:39.214852 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:39.214857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:39.214871 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:39.215145 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:39.215412 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:39.215414 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:39.215415 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:39.215635 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:39.216018 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:39.216023 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:39.216282 (+ 259us) client_negotiation.cc:769] Sending connection context
0806 00:56:39.216442 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 00:56:39.216738 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":146,"threads_started":1}
I20260806 00:56:39.219053 14489 heartbeater.cc:347] Connected to a master server at 127.27.77.254:41297
I20260806 00:56:39.219342 14489 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:39.219966 14489 heartbeater.cc:511] Master 127.27.77.254:41297 requested a full tablet report, sending...
I20260806 00:56:39.221167 13861 ts_manager.cc:194] Registered new tserver with Master: 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175)
I20260806 00:56:39.221843 13861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:41511
I20260806 00:56:39.232621 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:56:39.243187 13913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.235116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36174 (local address 127.27.77.254:41297)
0806 00:56:39.235292 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:39.235299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:39.235440 (+ 141us) server_negotiation.cc:407] Connection header received
0806 00:56:39.235560 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:39.235563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:39.235611 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:39.235688 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:39.236480 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.237110 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.238031 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.238281 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.238375 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.239848 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:39.239851 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:39.239853 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:39.239879 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:39.241482 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:39.241896 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:39.241899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:39.241900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:39.241983 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:39.242274 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:39.242276 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:39.242277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:39.242506 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:39.242612 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:39.242845 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 00:56:39.242951 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 00:56:39.246436 13861 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:56:39.249610 13861 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:56:39.264909 14513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.259642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41778 (local address 127.27.77.194:36549)
0806 00:56:39.260583 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:39.260596 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:39.260617 (+ 21us) server_negotiation.cc:407] Connection header received
0806 00:56:39.260691 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:39.260696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:39.260892 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:39.261064 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:39.261333 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.262401 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.263575 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.264173 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.264305 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.264461 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:39.264632 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 00:56:39.264730 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":158,"threads_started":1}
I20260806 00:56:39.264932 14506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.258619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:35109 (local address 127.0.0.1:60168)
0806 00:56:39.259041 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:39.259099 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:39.259241 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:39.260156 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:39.260165 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:39.260197 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:39.260408 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.260416 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.262196 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.262199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.263372 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.263382 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.264505 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.264508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.264603 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.264634 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 00:56:39.264680 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 00:56:39.264754 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":46,"thread_start_us":101,"threads_started":1}
I20260806 00:56:39.264922 14509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.259381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:36549 (local address 127.0.0.1:41778)
0806 00:56:39.260276 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:39.260309 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:39.260399 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:39.261110 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:39.261113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:39.261134 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:39.261213 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.261219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.262554 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.262560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.263419 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.263429 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.264306 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.264310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.264416 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.264453 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 00:56:39.264601 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 00:56:39.264730 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"thread_start_us":97,"threads_started":1}
I20260806 00:56:39.265455 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.258783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60168 (local address 127.27.77.193:35109)
0806 00:56:39.259517 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:39.259587 (+ 70us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:39.259614 (+ 27us) server_negotiation.cc:407] Connection header received
0806 00:56:39.259697 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:39.259703 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:39.259975 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:39.260589 (+ 614us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:39.260737 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.262045 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.263508 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.264363 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.265007 (+ 644us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.265149 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:39.265215 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 00:56:39.265309 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":126,"threads_started":1}
I20260806 00:56:39.266183 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.259028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:44175 (local address 127.0.0.1:48068)
0806 00:56:39.260128 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:39.260143 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:39.260241 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:39.260783 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:39.260786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:39.260805 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:39.260966 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.260976 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.262157 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.262163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.264522 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.264531 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.265806 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.265811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.265896 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.265918 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:56:39.265992 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 00:56:39.266060 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":968,"spinlock_wait_cycles":1408,"thread_start_us":93,"threads_started":1}
I20260806 00:56:39.267899 14511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.259500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.27.77.195:44175)
0806 00:56:39.260116 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:39.260129 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:39.260217 (+ 88us) server_negotiation.cc:407] Connection header received
0806 00:56:39.260381 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:39.260386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:39.260647 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:39.260807 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:39.261098 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.262001 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.264651 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.265466 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.265579 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.265736 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:39.266428 (+ 692us) server_negotiation.cc:299] Negotiation successful
0806 00:56:39.267751 (+ 1323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":184,"threads_started":1}
I20260806 00:56:39.268460 14205 tablet_service.cc:1467] Processing CreateTablet for tablet bfe046bc3c9f4f0e8492a9d45fe9088f (DEFAULT_TABLE table=test-table [id=95534f711cd84e5cbb8ae74cb832c487]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:56:39.268455 14026 tablet_service.cc:1467] Processing CreateTablet for tablet bfe046bc3c9f4f0e8492a9d45fe9088f (DEFAULT_TABLE table=test-table [id=95534f711cd84e5cbb8ae74cb832c487]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:56:39.269635 14205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfe046bc3c9f4f0e8492a9d45fe9088f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:39.269641 14026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfe046bc3c9f4f0e8492a9d45fe9088f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:39.270802 14394 tablet_service.cc:1467] Processing CreateTablet for tablet bfe046bc3c9f4f0e8492a9d45fe9088f (DEFAULT_TABLE table=test-table [id=95534f711cd84e5cbb8ae74cb832c487]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:56:39.271906 14394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfe046bc3c9f4f0e8492a9d45fe9088f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:39.276782 14514 tablet_bootstrap.cc:492] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68: Bootstrap starting.
I20260806 00:56:39.278877 14516 tablet_bootstrap.cc:492] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761: Bootstrap starting.
I20260806 00:56:39.279449 14515 tablet_bootstrap.cc:492] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: Bootstrap starting.
I20260806 00:56:39.279511 14514 tablet_bootstrap.cc:654] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:39.280485 14514 log.cc:824] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:39.281672 14516 tablet_bootstrap.cc:654] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:39.281694 14515 tablet_bootstrap.cc:654] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:39.282397 14514 tablet_bootstrap.cc:492] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68: No bootstrap required, opened a new log
I20260806 00:56:39.282552 14515 log.cc:824] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:39.282552 14516 log.cc:824] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:39.282619 14514 ts_tablet_manager.cc:1397] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 00:56:39.284472 14516 tablet_bootstrap.cc:492] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761: No bootstrap required, opened a new log
I20260806 00:56:39.284727 14516 ts_tablet_manager.cc:1397] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:56:39.284731 14515 tablet_bootstrap.cc:492] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: No bootstrap required, opened a new log
I20260806 00:56:39.284932 14515 ts_tablet_manager.cc:1397] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 00:56:39.286824 14514 raft_consensus.cc:348] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } }
I20260806 00:56:39.287110 14514 raft_consensus.cc:374] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:39.287168 14514 raft_consensus.cc:729] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace5781cc9314ccf8d24279b51e2cf68, State: Initialized, Role: FOLLOWER
I20260806 00:56:39.287724 14514 consensus_queue.cc:260] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } }
I20260806 00:56:39.287904 14516 raft_consensus.cc:348] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } }
I20260806 00:56:39.288187 14516 raft_consensus.cc:374] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:39.288261 14516 raft_consensus.cc:729] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ab771c8e62b4bfd83206f3ee3166761, State: Initialized, Role: FOLLOWER
I20260806 00:56:39.288693 14516 consensus_queue.cc:260] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } }
I20260806 00:56:39.289441 14514 ts_tablet_manager.cc:1428] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 00:56:39.289654 14516 ts_tablet_manager.cc:1428] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:56:39.289855 14297 heartbeater.cc:503] Master 127.27.77.254:41297 was elected leader, sending a full tablet report...
I20260806 00:56:39.290009 14489 heartbeater.cc:503] Master 127.27.77.254:41297 was elected leader, sending a full tablet report...
I20260806 00:56:39.291918 14515 raft_consensus.cc:348] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } }
I20260806 00:56:39.292186 14515 raft_consensus.cc:374] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:39.292264 14515 raft_consensus.cc:729] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a5681b349d4313a8a0ea9bec4d87fd, State: Initialized, Role: FOLLOWER
I20260806 00:56:39.292707 14515 consensus_queue.cc:260] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } }
I20260806 00:56:39.293569 14115 heartbeater.cc:503] Master 127.27.77.254:41297 was elected leader, sending a full tablet report...
I20260806 00:56:39.293754 14515 ts_tablet_manager.cc:1428] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: Time spent starting tablet: real 0.009s user 0.004s sys 0.004s
I20260806 00:56:39.394517 14524 raft_consensus.cc:482] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:56:39.394739 14524 raft_consensus.cc:504] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } }
I20260806 00:56:39.396188 14524 leader_election.cc:288] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90a5681b349d4313a8a0ea9bec4d87fd (127.27.77.193:35109), ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549)
I20260806 00:56:39.400035 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.396612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:60537 (local address 127.27.77.193:35109)
0806 00:56:39.396773 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:39.396778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:39.396801 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:56:39.396873 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:39.396877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:39.396943 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:39.397039 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:39.397453 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.397978 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.399031 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.399447 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.399603 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.399663 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:39.399839 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 00:56:39.399915 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:56:39.400035 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.396480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:35109 (local address 127.27.77.195:60537)
0806 00:56:39.396632 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:39.396651 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:39.396757 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:39.397183 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:39.397186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:39.397216 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:39.397315 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.397321 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.398104 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.398109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.398887 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.398896 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.399591 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.399594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.399706 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.399730 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 00:56:39.399823 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 00:56:39.399897 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 00:56:39.400511 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.396809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:36549 (local address 127.27.77.195:46771)
0806 00:56:39.397201 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:39.397213 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:39.397335 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:39.397634 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:39.397636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:39.397653 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:39.397774 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.397779 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.398503 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.398507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.399287 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.399296 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.400006 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.400010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.400114 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.400149 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 00:56:39.400266 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 00:56:39.400329 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":162,"threads_started":1}
I20260806 00:56:39.400532 14513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.396973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:46771 (local address 127.27.77.194:36549)
0806 00:56:39.397105 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:39.397110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:39.397306 (+ 196us) server_negotiation.cc:407] Connection header received
0806 00:56:39.397459 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:39.397463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:39.397515 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:39.397599 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:39.397897 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.398389 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.399419 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.399884 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.400010 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.400058 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:39.400308 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 00:56:39.400389 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:56:39.400795 14053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe046bc3c9f4f0e8492a9d45fe9088f" candidate_uuid: "0ab771c8e62b4bfd83206f3ee3166761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" is_pre_election: true
I20260806 00:56:39.401096 14053 raft_consensus.cc:2454] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ab771c8e62b4bfd83206f3ee3166761 in term 0.
I20260806 00:56:39.401280 14228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe046bc3c9f4f0e8492a9d45fe9088f" candidate_uuid: "0ab771c8e62b4bfd83206f3ee3166761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace5781cc9314ccf8d24279b51e2cf68" is_pre_election: true
I20260806 00:56:39.401592 14228 raft_consensus.cc:2454] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ab771c8e62b4bfd83206f3ee3166761 in term 0.
I20260806 00:56:39.401638 14323 leader_election.cc:302] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ab771c8e62b4bfd83206f3ee3166761, 90a5681b349d4313a8a0ea9bec4d87fd; no voters:
I20260806 00:56:39.401942 14524 raft_consensus.cc:2781] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:56:39.402030 14524 raft_consensus.cc:482] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:56:39.402063 14524 raft_consensus.cc:3037] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:39.403153 14524 raft_consensus.cc:504] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } }
I20260806 00:56:39.403510 14524 leader_election.cc:288] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [CANDIDATE]: Term 1 election: Requested vote from peers 90a5681b349d4313a8a0ea9bec4d87fd (127.27.77.193:35109), ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549)
I20260806 00:56:39.404091 14228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe046bc3c9f4f0e8492a9d45fe9088f" candidate_uuid: "0ab771c8e62b4bfd83206f3ee3166761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace5781cc9314ccf8d24279b51e2cf68"
I20260806 00:56:39.404099 14053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfe046bc3c9f4f0e8492a9d45fe9088f" candidate_uuid: "0ab771c8e62b4bfd83206f3ee3166761" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90a5681b349d4313a8a0ea9bec4d87fd"
I20260806 00:56:39.404239 14053 raft_consensus.cc:3037] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:39.404239 14228 raft_consensus.cc:3037] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:39.405735 14228 raft_consensus.cc:2454] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ab771c8e62b4bfd83206f3ee3166761 in term 1.
I20260806 00:56:39.405735 14053 raft_consensus.cc:2454] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ab771c8e62b4bfd83206f3ee3166761 in term 1.
I20260806 00:56:39.406033 14323 leader_election.cc:302] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ab771c8e62b4bfd83206f3ee3166761, ace5781cc9314ccf8d24279b51e2cf68; no voters:
I20260806 00:56:39.406210 14524 raft_consensus.cc:2781] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:56:39.406481 14524 raft_consensus.cc:686] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [term 1 LEADER]: Becoming Leader. State: Replica: 0ab771c8e62b4bfd83206f3ee3166761, State: Running, Role: LEADER
I20260806 00:56:39.406840 14524 consensus_queue.cc:237] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } }
I20260806 00:56:39.410353 13860 catalog_manager.cc:5495] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195). New cstate: current_term: 1 leader_uuid: "0ab771c8e62b4bfd83206f3ee3166761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } health_report { overall_health: HEALTHY } } }
W20260806 00:56:39.452376 14491 tablet.cc:2369] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:56:39.456043 14511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.451914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.27.77.195:44175)
0806 00:56:39.452252 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:39.452257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:39.452276 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:56:39.452363 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:39.452369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:39.452447 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:39.452582 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:56:39.452878 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.453472 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.454417 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.454669 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.454774 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.455124 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:56:39.455611 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:56:39.455713 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:39.455834 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 00:56:39.455897 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
W20260806 00:56:39.462571 14298 tablet.cc:2369] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:56:39.463088 14228 raft_consensus.cc:1261] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68 [term 1 FOLLOWER]: Refusing update from remote peer 0ab771c8e62b4bfd83206f3ee3166761: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:56:39.463088 14053 raft_consensus.cc:1261] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Refusing update from remote peer 0ab771c8e62b4bfd83206f3ee3166761: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:56:39.464015 14524 consensus_queue.cc:1035] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:56:39.464258 14528 consensus_queue.cc:1035] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 00:56:39.471662 14116 tablet.cc:2369] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:56:39.483448 13857 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:56:39.483700 13857 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:56:39.486418 13857 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: Sending DeleteTablet for 3 replicas of tablet bfe046bc3c9f4f0e8492a9d45fe9088f
I20260806 00:56:39.487380 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 13946
I20260806 00:56:39.487396 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet bfe046bc3c9f4f0e8492a9d45fe9088f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:56:39 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:48068
I20260806 00:56:39.487390 14205 tablet_service.cc:1514] Processing DeleteTablet for tablet bfe046bc3c9f4f0e8492a9d45fe9088f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:56:39 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:41778
I20260806 00:56:39.492836 14546 tablet_replica.cc:330] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68: stopping tablet replica
I20260806 00:56:39.493302 14546 raft_consensus.cc:2229] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:56:39.493670 14546 raft_consensus.cc:2258] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:56:39.494244 14547 tablet_replica.cc:330] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761: stopping tablet replica
I20260806 00:56:39.494573 14547 raft_consensus.cc:2229] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:56:39.495039 14547 raft_consensus.cc:2258] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:56:39.496001 14546 ts_tablet_manager.cc:1905] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:56:39.496544 14547 ts_tablet_manager.cc:1905] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:56:39.499358 14547 ts_tablet_manager.cc:1918] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:56:39.499358 14546 ts_tablet_manager.cc:1918] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:56:39.499488 14547 log.cc:1196] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/bfe046bc3c9f4f0e8492a9d45fe9088f
I20260806 00:56:39.499655 14546 log.cc:1196] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/bfe046bc3c9f4f0e8492a9d45fe9088f
I20260806 00:56:39.499893 14547 ts_tablet_manager.cc:1939] T bfe046bc3c9f4f0e8492a9d45fe9088f P 0ab771c8e62b4bfd83206f3ee3166761: Deleting consensus metadata
I20260806 00:56:39.500155 14546 ts_tablet_manager.cc:1939] T bfe046bc3c9f4f0e8492a9d45fe9088f P ace5781cc9314ccf8d24279b51e2cf68: Deleting consensus metadata
I20260806 00:56:39.501358 13840 catalog_manager.cc:4841] TS ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549): tablet bfe046bc3c9f4f0e8492a9d45fe9088f (table test-table [id=95534f711cd84e5cbb8ae74cb832c487]) successfully deleted
I20260806 00:56:39.501358 13843 catalog_manager.cc:4841] TS 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175): tablet bfe046bc3c9f4f0e8492a9d45fe9088f (table test-table [id=95534f711cd84e5cbb8ae74cb832c487]) successfully deleted
W20260806 00:56:39.501706 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:35109 (error 108)
I20260806 00:56:39.501777 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 14122
I20260806 00:56:39.507619 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.507188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:35109 (local address 127.0.0.1:60172)
0806 00:56:39.507405 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:39.507486 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:35109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260806 00:56:39.507979 13840 catalog_manager.cc:4584] TS 90a5681b349d4313a8a0ea9bec4d87fd (127.27.77.193:35109): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bfe046bc3c9f4f0e8492a9d45fe9088f: Network error: Client connection negotiation failed: client connection to 127.27.77.193:35109: connect: Connection refused (error 111)
I20260806 00:56:39.514420 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 14303
I20260806 00:56:39.528188 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 13818
I20260806 00:56:39.542847 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:41297
--webserver_interface=127.27.77.254
--webserver_port=42483
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:39101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:41297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:39.658675 14551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:39.659003 14551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:39.659089 14551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:39.663141 14551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:56:39.663244 14551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:39.663282 14551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:39.663336 14551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:56:39.663372 14551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:56:39.669162 14551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:39101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:41297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:41297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=42483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:39.670286 14551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:39.671352 14551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:39.686295 14563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:39.686295 14564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:39.686295 14567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:39.686575 14551 server_base.cc:1034] running on GCE node
I20260806 00:56:39.687245 14551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:39.688542 14551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:39.689739 14551 hybrid_clock.cc:648] HybridClock initialized: now 1785977799689727 us; error 27 us; skew 500 ppm
Aug 06 00:56:39 dist-test-slave-sb2z krb5kdc[13799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977799, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:39.692693 14551 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:56:39.693723 14551 webserver.cc:460] Webserver started at http://127.27.77.254:42483/ using document root <none> and password file <none>
I20260806 00:56:39.694283 14551 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:39.694358 14551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:39.698410 14551 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260806 00:56:39.700642 14578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:39.702045 14551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:56:39.702220 14551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "74621d8113284ab7abe2a8be5ab6ca41"
format_stamp: "Formatted at 2026-08-06 00:56:38 on dist-test-slave-sb2z"
server_key: "94af1c229611d3d503461233998305a7"
server_key_iv: "ff7fb31f8ca99ba1f402d23284928e3c"
server_key_version: "encryptionkey@0"
I20260806 00:56:39.702852 14551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:39.718665 14551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:39.721968 14551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:39.722208 14551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:39.729712 14551 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:41297
I20260806 00:56:39.730019 14654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:41297 every 8 connection(s)
I20260806 00:56:39.730911 14551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:56:39.735993 14656 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:56:39.737835 14656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: Bootstrap starting.
I20260806 00:56:39.741014 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 14551
I20260806 00:56:39.741544 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:35109
--local_ip_for_outbound_sockets=127.27.77.193
--tserver_master_addrs=127.27.77.254:41297
--webserver_port=35169
--webserver_interface=127.27.77.193
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:39101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:56:39.757499 14656 log.cc:824] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:39.767241 14656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:56:39.767725 14656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: Bootstrap complete.
I20260806 00:56:39.772203 14656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } }
I20260806 00:56:39.772635 14656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74621d8113284ab7abe2a8be5ab6ca41, State: Initialized, Role: FOLLOWER
I20260806 00:56:39.773167 14656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } }
I20260806 00:56:39.773290 14656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:56:39.773376 14656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:56:39.773502 14656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:56:39.775573 14656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } }
I20260806 00:56:39.775946 14656 leader_election.cc:302] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74621d8113284ab7abe2a8be5ab6ca41; no voters:
I20260806 00:56:39.776320 14656 leader_election.cc:288] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 00:56:39.776412 14660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 00:56:39.776646 14660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 2 LEADER]: Becoming Leader. State: Replica: 74621d8113284ab7abe2a8be5ab6ca41, State: Running, Role: LEADER
I20260806 00:56:39.777050 14660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } }
I20260806 00:56:39.777657 14656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:56:39.778606 14662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "74621d8113284ab7abe2a8be5ab6ca41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } } }
I20260806 00:56:39.778757 14662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:39.779016 14663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74621d8113284ab7abe2a8be5ab6ca41. Latest consensus state: current_term: 2 leader_uuid: "74621d8113284ab7abe2a8be5ab6ca41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } } }
I20260806 00:56:39.779098 14663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:39.782549 14681 catalog_manager.cc:1259] Loaded cluster ID: ff25458b51a84697a11be8b4b137840f
I20260806 00:56:39.782634 14681 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: loading cluster ID for follower catalog manager: success
W20260806 00:56:39.783448 14681 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260806 00:56:39.783567 14681 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260806 00:56:39.783630 14671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:56:39.785645 14671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:56:39.786084 14671 catalog_manager.cc:1259] Loaded cluster ID: ff25458b51a84697a11be8b4b137840f
I20260806 00:56:39.786162 14671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:56:39.787098 14671 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 00:56:39.789891 14671 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 00:56:39.794407 14671 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:56:39.795159 14671 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: Loaded TSK: 0
I20260806 00:56:39.796226 14671 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:56:39.874571 14658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:39.874845 14658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:39.874889 14658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:39.879256 14658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:39.879320 14658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:39.879411 14658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:56:39.885373 14658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:39101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:35109
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=35169
--tserver_master_addrs=127.27.77.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:39.886439 14658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:39.887455 14658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:39.897789 14691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:39.897787 14690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:39.898084 14658 server_base.cc:1034] running on GCE node
W20260806 00:56:39.898115 14693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:39.898610 14658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:39.899376 14658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:39.900583 14658 hybrid_clock.cc:648] HybridClock initialized: now 1785977799900546 us; error 45 us; skew 500 ppm
Aug 06 00:56:39 dist-test-slave-sb2z krb5kdc[13799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977799, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:39.904078 14658 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:56:39.905035 14658 webserver.cc:460] Webserver started at http://127.27.77.193:35169/ using document root <none> and password file <none>
I20260806 00:56:39.905620 14658 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:39.905742 14658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:39.910091 14658 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:56:39.912652 14703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:39.914011 14658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:56:39.914201 14658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "90a5681b349d4313a8a0ea9bec4d87fd"
format_stamp: "Formatted at 2026-08-06 00:56:38 on dist-test-slave-sb2z"
server_key: "34ed7a9f34fac7e3a891f068f5c0771c"
server_key_iv: "7a22f727c9686fa7af94a57ea5720d6e"
server_key_version: "encryptionkey@0"
I20260806 00:56:39.914772 14658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:39.938649 14658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:39.942349 14658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:39.942619 14658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:39.943331 14658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:39.944510 14712 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 00:56:39.948840 14658 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 00:56:39.948935 14658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260806 00:56:39.949080 14658 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 00:56:39.950941 14658 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 00:56:39.951016 14658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 00:56:39.951365 14712 tablet_bootstrap.cc:492] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: Bootstrap starting.
I20260806 00:56:39.962172 14658 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:35109
I20260806 00:56:39.962318 14861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:35109 every 8 connection(s)
I20260806 00:56:39.963256 14658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:56:39.970454 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 14658
I20260806 00:56:39.970959 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:36549
--local_ip_for_outbound_sockets=127.27.77.194
--tserver_master_addrs=127.27.77.254:41297
--webserver_port=45961
--webserver_interface=127.27.77.194
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:39101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:56:39.972609 14712 log.cc:824] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:56:39 dist-test-slave-sb2z krb5kdc[13799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977799, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:39.982177 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.965471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:55819 (local address 127.27.77.254:41297)
0806 00:56:39.965891 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:39.965911 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:39.966223 (+ 312us) server_negotiation.cc:407] Connection header received
0806 00:56:39.967363 (+ 1140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:39.967379 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:39.967803 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:39.968197 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:39.969356 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.971055 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.972726 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.973310 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.973502 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.976487 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:39.976493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:39.976506 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:39.976535 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:39.978471 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:39.979156 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:39.979166 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:39.979173 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:39.979305 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:39.979759 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:39.979764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:39.979765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:39.980276 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:39.980430 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:39.981006 (+ 576us) server_negotiation.cc:299] Negotiation successful
0806 00:56:39.981275 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":128,"threads_started":1}
I20260806 00:56:39.982183 14866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.965347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:55819)
0806 00:56:39.966024 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:39.966080 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:39.966904 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:39.968268 (+ 1364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:39.968280 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:39.968648 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:39.969144 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.969160 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.971229 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.971233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.972555 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.972569 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.973472 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.973476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.973674 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.974447 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:39.974473 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:39.976266 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:39.978603 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:39.978610 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:39.978624 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:39.978988 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:39.979428 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:39.979431 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:39.979433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:39.979657 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:39.980401 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:39.980407 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:39.980667 (+ 260us) client_negotiation.cc:769] Sending connection context
0806 00:56:39.980852 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 00:56:39.981231 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":179,"threads_started":1}
I20260806 00:56:39.983479 14712 tablet_bootstrap.cc:492] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:56:39.984007 14712 tablet_bootstrap.cc:492] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: Bootstrap complete.
I20260806 00:56:39.985080 14862 heartbeater.cc:347] Connected to a master server at 127.27.77.254:41297
I20260806 00:56:39.985311 14862 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:39.985772 14862 heartbeater.cc:511] Master 127.27.77.254:41297 requested a full tablet report, sending...
I20260806 00:56:39.985823 14712 ts_tablet_manager.cc:1397] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: Time spent bootstrapping tablet: real 0.035s user 0.024s sys 0.008s
I20260806 00:56:39.988627 14603 ts_manager.cc:194] Registered new tserver with Master: 90a5681b349d4313a8a0ea9bec4d87fd (127.27.77.193:35109)
I20260806 00:56:39.988958 14712 raft_consensus.cc:348] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } }
I20260806 00:56:39.989512 14712 raft_consensus.cc:729] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a5681b349d4313a8a0ea9bec4d87fd, State: Initialized, Role: FOLLOWER
I20260806 00:56:39.990000 14712 consensus_queue.cc:260] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } }
I20260806 00:56:39.991079 14712 ts_tablet_manager.cc:1428] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260806 00:56:39.992600 14603 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:55819
I20260806 00:56:39.995213 14862 heartbeater.cc:503] Master 127.27.77.254:41297 was elected leader, sending a full tablet report...
Aug 06 00:56:39 dist-test-slave-sb2z krb5kdc[13799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977799, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for kudu/127.27.77.193@KRBTEST.COM
I20260806 00:56:40.003659 14871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.991940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:35109 (local address 127.0.0.1:60182)
0806 00:56:39.992521 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:39.992575 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:39.992711 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:39.993292 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:39.993298 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:39.993815 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:39.993980 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.993989 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.995307 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.995311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.996330 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:39.996342 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.997213 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.997216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:39.997341 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.997990 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:39.998008 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:39.999664 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:40.001674 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:40.001683 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:40.001689 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:40.001981 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:40.002329 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:40.002332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:40.002334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:40.002430 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:40.002924 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:40.002930 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:40.003060 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 00:56:40.003210 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 00:56:40.003411 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":121,"threads_started":1}
I20260806 00:56:40.003921 14872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:39.992029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60182 (local address 127.27.77.193:35109)
0806 00:56:39.992774 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:39.992789 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:39.992815 (+ 26us) server_negotiation.cc:407] Connection header received
0806 00:56:39.992904 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:39.992910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:39.993113 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:39.993273 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:39.994121 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.995144 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.996474 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:39.997096 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:39.997186 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:39.999801 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:39.999811 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:39.999821 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:39.999843 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:40.001558 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:40.002128 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:40.002133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:40.002134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:40.002225 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:40.002531 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:40.002534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:40.002535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:40.002804 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:40.002924 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:40.003432 (+ 508us) server_negotiation.cc:299] Negotiation successful
0806 00:56:40.003681 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":90,"threads_started":1}
I20260806 00:56:40.004957 14771 tablet_service.cc:1514] Processing DeleteTablet for tablet bfe046bc3c9f4f0e8492a9d45fe9088f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:56:39 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:60182
I20260806 00:56:40.006068 14873 tablet_replica.cc:330] stopping tablet replica
I20260806 00:56:40.006400 14873 raft_consensus.cc:2229] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:56:40.006669 14873 raft_consensus.cc:2258] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:56:40.008157 14873 ts_tablet_manager.cc:1905] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:56:40.011891 14873 ts_tablet_manager.cc:1918] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:56:40.012017 14873 log.cc:1196] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal/wals/bfe046bc3c9f4f0e8492a9d45fe9088f
I20260806 00:56:40.012338 14873 ts_tablet_manager.cc:1939] T bfe046bc3c9f4f0e8492a9d45fe9088f P 90a5681b349d4313a8a0ea9bec4d87fd: Deleting consensus metadata
I20260806 00:56:40.014127 14582 catalog_manager.cc:4841] TS 90a5681b349d4313a8a0ea9bec4d87fd (127.27.77.193:35109): tablet bfe046bc3c9f4f0e8492a9d45fe9088f (table test-table [id=95534f711cd84e5cbb8ae74cb832c487]) successfully deleted
W20260806 00:56:40.098651 14867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:40.098903 14867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:40.098946 14867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:40.103047 14867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:40.103113 14867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:40.103174 14867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:56:40.108562 14867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:39101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:36549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=45961
--tserver_master_addrs=127.27.77.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:40.109567 14867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:40.110612 14867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:40.120319 14883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:40.120291 14884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:40.120291 14887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:40.120689 14867 server_base.cc:1034] running on GCE node
I20260806 00:56:40.121146 14867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:40.121799 14867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:40.122982 14867 hybrid_clock.cc:648] HybridClock initialized: now 1785977800122949 us; error 49 us; skew 500 ppm
Aug 06 00:56:40 dist-test-slave-sb2z krb5kdc[13799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977800, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:40.125878 14867 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:56:40.126794 14867 webserver.cc:460] Webserver started at http://127.27.77.194:45961/ using document root <none> and password file <none>
I20260806 00:56:40.127367 14867 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:40.127485 14867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:40.131414 14867 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:56:40.133841 14894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:40.135313 14867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:56:40.135468 14867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "ace5781cc9314ccf8d24279b51e2cf68"
format_stamp: "Formatted at 2026-08-06 00:56:38 on dist-test-slave-sb2z"
server_key: "8322a617b3a5648aec8b13549e6c80a0"
server_key_iv: "fa84f641a1e29ccecd6965f17e0a7c5a"
server_key_version: "encryptionkey@0"
I20260806 00:56:40.136153 14867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:40.154276 14867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:40.157776 14867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:40.158070 14867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:40.158864 14867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:40.159936 14867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:40.160025 14867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:40.160110 14867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:40.160169 14867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:40.170658 14867 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:36549
I20260806 00:56:40.170665 15046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:36549 every 8 connection(s)
I20260806 00:56:40.171901 14867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:56:40.181136 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 14867
I20260806 00:56:40.181643 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:44175
--local_ip_for_outbound_sockets=127.27.77.195
--tserver_master_addrs=127.27.77.254:41297
--webserver_port=43673
--webserver_interface=127.27.77.195
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:39101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:56:40 dist-test-slave-sb2z krb5kdc[13799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977800, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:40.187182 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.173883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:37807 (local address 127.27.77.254:41297)
0806 00:56:40.174033 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:40.174037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:40.174924 (+ 887us) server_negotiation.cc:407] Connection header received
0806 00:56:40.176442 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:40.176445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:40.176508 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:40.176617 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:40.177976 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.178685 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.180020 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.180285 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.180398 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:40.183097 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:40.183101 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:40.183104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:40.183141 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:40.184917 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:40.185503 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:40.185512 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:40.185513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:40.185597 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:40.185983 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:40.185988 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:40.185990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:40.186193 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:40.186309 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:40.186852 (+ 543us) server_negotiation.cc:299] Negotiation successful
0806 00:56:40.187004 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:56:40.188160 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.174268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:37807)
0806 00:56:40.174767 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:40.174811 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:40.176210 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:40.176800 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:40.176807 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:40.177187 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:40.177811 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:40.177827 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.178831 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.178835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:40.179887 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:40.179898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.180459 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.180463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:40.180636 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:40.181328 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:40.181346 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:40.182901 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:40.185051 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:40.185058 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:40.185076 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:40.185378 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:40.185712 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:40.185715 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:40.185717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:40.185878 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:40.186310 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:40.186319 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:40.186600 (+ 281us) client_negotiation.cc:769] Sending connection context
0806 00:56:40.186810 (+ 210us) client_negotiation.cc:240] Negotiation successful
0806 00:56:40.187180 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":132,"threads_started":1}
I20260806 00:56:40.189497 15047 heartbeater.cc:347] Connected to a master server at 127.27.77.254:41297
I20260806 00:56:40.189792 15047 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:40.190323 15047 heartbeater.cc:511] Master 127.27.77.254:41297 requested a full tablet report, sending...
I20260806 00:56:40.191512 14603 ts_manager.cc:194] Registered new tserver with Master: ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549)
I20260806 00:56:40.192481 14603 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:37807
W20260806 00:56:40.299899 15050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:40.300144 15050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:40.300189 15050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:40.304241 15050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:40.304303 15050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:40.304363 15050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:56:40.309785 15050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:39101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:44175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=43673
--tserver_master_addrs=127.27.77.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:40.310812 15050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:40.311905 15050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:40.321969 15059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:40.321969 15062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:40.322180 15050 server_base.cc:1034] running on GCE node
W20260806 00:56:40.322069 15057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:40.322924 15050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:40.323611 15050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:40.324791 15050 hybrid_clock.cc:648] HybridClock initialized: now 1785977800324779 us; error 30 us; skew 500 ppm
Aug 06 00:56:40 dist-test-slave-sb2z krb5kdc[13799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977800, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:40.327761 15050 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:56:40.328603 15050 webserver.cc:460] Webserver started at http://127.27.77.195:43673/ using document root <none> and password file <none>
I20260806 00:56:40.329141 15050 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:40.329216 15050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:40.333243 15050 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:56:40.335634 15073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:40.337343 15050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:56:40.337494 15050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "0ab771c8e62b4bfd83206f3ee3166761"
format_stamp: "Formatted at 2026-08-06 00:56:39 on dist-test-slave-sb2z"
server_key: "cf482e91ee8011546b9132c1649717e3"
server_key_iv: "faea17daf0cabeaa4c737ee384392108"
server_key_version: "encryptionkey@0"
I20260806 00:56:40.338223 15050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:40.352628 15050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:40.356532 15050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:40.356777 15050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:40.357458 15050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:40.358367 15050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:40.358443 15050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:40.358523 15050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:40.358577 15050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:40.368177 15050 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:44175
I20260806 00:56:40.368238 15231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:44175 every 8 connection(s)
I20260806 00:56:40.369270 15050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:56:40.369858 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 15050
I20260806 00:56:40.380244 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.370702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36186 (local address 127.27.77.254:41297)
0806 00:56:40.370865 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:40.370872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:40.371155 (+ 283us) server_negotiation.cc:407] Connection header received
0806 00:56:40.371354 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:40.371358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:40.371412 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:40.371559 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:40.372561 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.373301 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.374236 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.374471 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.374554 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:40.376264 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:40.376267 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:40.376269 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:40.376292 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:40.378078 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:40.378618 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:40.378621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:40.378622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:40.378730 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:40.379135 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:40.379137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:40.379139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:40.379313 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:40.379477 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:40.379824 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 00:56:40.379995 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 00:56:40 dist-test-slave-sb2z krb5kdc[13799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977800, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:40.385558 15238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.371412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:59929 (local address 127.27.77.254:41297)
0806 00:56:40.373132 (+ 1720us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:40.373137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:40.373152 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:56:40.374057 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:40.374060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:40.374114 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:40.374207 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:40.376133 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.376762 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.378480 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.378880 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.378992 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:40.381887 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:40.381890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:40.381892 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:40.381915 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:40.383471 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:40.384085 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:40.384088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:40.384089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:40.384170 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:40.384492 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:40.384495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:40.384496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:40.384697 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:40.384780 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:40.385316 (+ 536us) server_negotiation.cc:299] Negotiation successful
0806 00:56:40.385423 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1643,"thread_start_us":135,"threads_started":1}
I20260806 00:56:40.386387 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.372037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:59929)
0806 00:56:40.372588 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:40.372631 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:40.373854 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:40.374877 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:40.374889 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:40.375406 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:40.375971 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:40.375985 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.376904 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.376909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:40.378355 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:40.378365 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.379066 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.379073 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:40.379267 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:40.379905 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:40.379928 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:40.381719 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:40.383646 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:40.383651 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:40.383664 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:40.383919 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:40.384256 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:40.384258 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:40.384259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:40.384395 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:40.384796 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:40.384802 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:40.385070 (+ 268us) client_negotiation.cc:769] Sending connection context
0806 00:56:40.385225 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 00:56:40.385484 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":146,"threads_started":1}
I20260806 00:56:40.387583 15232 heartbeater.cc:347] Connected to a master server at 127.27.77.254:41297
I20260806 00:56:40.387857 15232 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:40.388383 15232 heartbeater.cc:511] Master 127.27.77.254:41297 requested a full tablet report, sending...
I20260806 00:56:40.389489 14603 ts_manager.cc:194] Registered new tserver with Master: 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175)
I20260806 00:56:40.390120 14603 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:59929
I20260806 00:56:40.394294 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:56:40.398540 15238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.394906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36198 (local address 127.27.77.254:41297)
0806 00:56:40.395047 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:40.395051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:40.395233 (+ 182us) server_negotiation.cc:407] Connection header received
0806 00:56:40.395366 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:40.395368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:40.395465 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:40.395682 (+ 217us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:56:40.395905 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.396455 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.397276 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.397508 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.397592 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:40.397847 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:56:40.398227 (+ 380us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:56:40.398320 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:40.398382 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 00:56:40.398435 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:56:40.399200 14603 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:56:40.402818 14603 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:56:40.412359 14771 tablet_service.cc:1467] Processing CreateTablet for tablet 85da5d55e09a4a82a37b90f36ab00054 (DEFAULT_TABLE table=test-table [id=8abb4cb99f9341fdaa312af5eb4e3d91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:56:40.413110 14771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85da5d55e09a4a82a37b90f36ab00054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:40.416373 14871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.410350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:44175 (local address 127.0.0.1:48080)
0806 00:56:40.410517 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:40.410531 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:40.410603 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:40.411804 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:40.411807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:40.411823 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:40.411953 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:40.411964 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.413630 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.413635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:40.414564 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:40.414574 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.415901 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.415905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:40.416016 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:40.416064 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 00:56:40.416151 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 00:56:40.416231 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 00:56:40.416478 14712 tablet_bootstrap.cc:492] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd: Bootstrap starting.
I20260806 00:56:40.416658 15245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.410768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48080 (local address 127.27.77.195:44175)
0806 00:56:40.411161 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:40.411170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:40.411212 (+ 42us) server_negotiation.cc:407] Connection header received
0806 00:56:40.411302 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:40.411308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:40.411624 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:40.411832 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:40.412090 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.413487 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.414710 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.415719 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.415884 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:40.416070 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:40.416207 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 00:56:40.416319 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":179,"threads_started":1}
I20260806 00:56:40.417239 15247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.411251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:36549 (local address 127.0.0.1:41780)
0806 00:56:40.411556 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:40.411571 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:40.411669 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:40.412672 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:40.412675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:40.412694 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:40.412810 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:40.412817 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.414306 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.414310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:40.415393 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:40.415405 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.416835 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.416839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:40.416930 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:40.416948 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 00:56:40.417040 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 00:56:40.417113 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":133,"threads_started":1}
I20260806 00:56:40.417439 14712 tablet_bootstrap.cc:654] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:40.417516 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.411491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41780 (local address 127.27.77.194:36549)
0806 00:56:40.411957 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:40.411970 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:40.411995 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:56:40.412078 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:40.412084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:40.412305 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:40.412469 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:40.412956 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.414145 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.415574 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.416534 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.416638 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:40.416772 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:40.417076 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 00:56:40.417175 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":115,"threads_started":1}
I20260806 00:56:40.420542 14712 tablet_bootstrap.cc:492] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd: No bootstrap required, opened a new log
I20260806 00:56:40.420657 14712 ts_tablet_manager.cc:1397] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 00:56:40.420411 14962 tablet_service.cc:1467] Processing CreateTablet for tablet 85da5d55e09a4a82a37b90f36ab00054 (DEFAULT_TABLE table=test-table [id=8abb4cb99f9341fdaa312af5eb4e3d91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:56:40.420435 15140 tablet_service.cc:1467] Processing CreateTablet for tablet 85da5d55e09a4a82a37b90f36ab00054 (DEFAULT_TABLE table=test-table [id=8abb4cb99f9341fdaa312af5eb4e3d91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:56:40.421190 14712 raft_consensus.cc:348] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:40.421336 14712 raft_consensus.cc:374] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:40.421378 14712 raft_consensus.cc:729] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a5681b349d4313a8a0ea9bec4d87fd, State: Initialized, Role: FOLLOWER
I20260806 00:56:40.421554 14962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85da5d55e09a4a82a37b90f36ab00054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:40.421520 14712 consensus_queue.cc:260] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:40.421592 15140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85da5d55e09a4a82a37b90f36ab00054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:40.421823 14712 ts_tablet_manager.cc:1428] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:56:40.429598 15251 tablet_bootstrap.cc:492] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761: Bootstrap starting.
I20260806 00:56:40.430647 15252 tablet_bootstrap.cc:492] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68: Bootstrap starting.
I20260806 00:56:40.432585 15251 tablet_bootstrap.cc:654] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:40.433421 15252 tablet_bootstrap.cc:654] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:40.433408 15251 log.cc:824] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:40.434285 15252 log.cc:824] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:40.435457 15251 tablet_bootstrap.cc:492] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761: No bootstrap required, opened a new log
I20260806 00:56:40.435719 15251 ts_tablet_manager.cc:1397] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:56:40.436246 15252 tablet_bootstrap.cc:492] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68: No bootstrap required, opened a new log
I20260806 00:56:40.436453 15252 ts_tablet_manager.cc:1397] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260806 00:56:40.439018 15251 raft_consensus.cc:348] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:40.439283 15251 raft_consensus.cc:374] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:40.439368 15251 raft_consensus.cc:729] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ab771c8e62b4bfd83206f3ee3166761, State: Initialized, Role: FOLLOWER
I20260806 00:56:40.439844 15251 consensus_queue.cc:260] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:40.440021 15252 raft_consensus.cc:348] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:40.440270 15252 raft_consensus.cc:374] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:40.440337 15252 raft_consensus.cc:729] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace5781cc9314ccf8d24279b51e2cf68, State: Initialized, Role: FOLLOWER
I20260806 00:56:40.440671 15232 heartbeater.cc:503] Master 127.27.77.254:41297 was elected leader, sending a full tablet report...
I20260806 00:56:40.440900 15251 ts_tablet_manager.cc:1428] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:56:40.440877 15252 consensus_queue.cc:260] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:40.441623 15047 heartbeater.cc:503] Master 127.27.77.254:41297 was elected leader, sending a full tablet report...
I20260806 00:56:40.441879 15252 ts_tablet_manager.cc:1428] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260806 00:56:40.464702 14863 tablet.cc:2369] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:56:40.620427 15235 tablet.cc:2369] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:56:40.673249 15048 tablet.cc:2369] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:56:40.741811 14870 raft_consensus.cc:482] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:56:40.742062 14870 raft_consensus.cc:504] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:40.743412 14870 leader_election.cc:288] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175), ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549)
I20260806 00:56:40.749841 15245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.743859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:54477 (local address 127.27.77.195:44175)
0806 00:56:40.744062 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:40.744067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:40.744395 (+ 328us) server_negotiation.cc:407] Connection header received
0806 00:56:40.744632 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:40.744639 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:40.744700 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:40.744782 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:40.745239 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.746037 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.748343 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.748965 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.749109 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:40.749167 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:40.749616 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 00:56:40.749704 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:56:40.749841 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.743985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:44007 (local address 127.27.77.194:36549)
0806 00:56:40.744111 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:40.744115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:40.744606 (+ 491us) server_negotiation.cc:407] Connection header received
0806 00:56:40.744831 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:40.744835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:40.744899 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:40.744954 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:40.745365 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.746127 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.748343 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.748965 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.749108 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:40.749174 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:40.749616 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 00:56:40.749702 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:56:40.749948 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.743798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:36549 (local address 127.27.77.193:44007)
0806 00:56:40.744479 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:40.744494 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:40.744687 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:40.745098 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:40.745101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:40.745118 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:40.745222 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:40.745228 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.746280 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.746283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:40.748166 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:40.748179 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.749133 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.749138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:40.749401 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:40.749487 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 00:56:40.749597 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 00:56:40.749749 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":552,"thread_start_us":85,"threads_started":1}
I20260806 00:56:40.750219 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.743793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:44175 (local address 127.27.77.193:54477)
0806 00:56:40.744301 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:40.744317 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:40.744456 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:40.744920 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:40.744926 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:40.744960 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:40.745103 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:40.745110 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.746179 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.746186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:40.748166 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:40.748179 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.749133 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.749138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:40.749405 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:40.749487 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 00:56:40.749579 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 00:56:40.749749 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":395,"thread_start_us":76,"threads_started":1}
I20260806 00:56:40.750624 14985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85da5d55e09a4a82a37b90f36ab00054" candidate_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace5781cc9314ccf8d24279b51e2cf68" is_pre_election: true
I20260806 00:56:40.750900 15168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85da5d55e09a4a82a37b90f36ab00054" candidate_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ab771c8e62b4bfd83206f3ee3166761" is_pre_election: true
I20260806 00:56:40.750981 14985 raft_consensus.cc:2454] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90a5681b349d4313a8a0ea9bec4d87fd in term 0.
I20260806 00:56:40.751202 15168 raft_consensus.cc:2454] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90a5681b349d4313a8a0ea9bec4d87fd in term 0.
I20260806 00:56:40.751597 14706 leader_election.cc:302] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90a5681b349d4313a8a0ea9bec4d87fd, ace5781cc9314ccf8d24279b51e2cf68; no voters:
I20260806 00:56:40.751926 14870 raft_consensus.cc:2781] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:56:40.752046 14870 raft_consensus.cc:482] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:56:40.752101 14870 raft_consensus.cc:3037] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:40.753212 14870 raft_consensus.cc:504] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:40.753578 14870 leader_election.cc:288] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 1 election: Requested vote from peers 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175), ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549)
I20260806 00:56:40.754037 15168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85da5d55e09a4a82a37b90f36ab00054" candidate_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ab771c8e62b4bfd83206f3ee3166761"
I20260806 00:56:40.754060 14985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85da5d55e09a4a82a37b90f36ab00054" candidate_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace5781cc9314ccf8d24279b51e2cf68"
I20260806 00:56:40.754196 15168 raft_consensus.cc:3037] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:40.754211 14985 raft_consensus.cc:3037] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:40.755304 15168 raft_consensus.cc:2454] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90a5681b349d4313a8a0ea9bec4d87fd in term 1.
I20260806 00:56:40.755350 14985 raft_consensus.cc:2454] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90a5681b349d4313a8a0ea9bec4d87fd in term 1.
I20260806 00:56:40.755724 14708 leader_election.cc:302] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ab771c8e62b4bfd83206f3ee3166761, 90a5681b349d4313a8a0ea9bec4d87fd; no voters:
I20260806 00:56:40.755923 14870 raft_consensus.cc:2781] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:56:40.756203 14870 raft_consensus.cc:686] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 LEADER]: Becoming Leader. State: Replica: 90a5681b349d4313a8a0ea9bec4d87fd, State: Running, Role: LEADER
I20260806 00:56:40.756484 14870 consensus_queue.cc:237] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:40.759804 14603 catalog_manager.cc:5495] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 90a5681b349d4313a8a0ea9bec4d87fd (127.27.77.193). New cstate: current_term: 1 leader_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } health_report { overall_health: HEALTHY } } }
I20260806 00:56:40.786407 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.782768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60190 (local address 127.27.77.193:35109)
0806 00:56:40.783079 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:40.783086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:40.783100 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:56:40.783190 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:40.783193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:40.783246 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:40.783355 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:56:40.783621 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.784194 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.784971 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:40.785234 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:40.785324 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:40.785563 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:56:40.786002 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:56:40.786122 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:40.786214 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 00:56:40.786264 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":131,"threads_started":1}
I20260806 00:56:40.790639 14985 raft_consensus.cc:1261] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68 [term 1 FOLLOWER]: Refusing update from remote peer 90a5681b349d4313a8a0ea9bec4d87fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:56:40.790639 15168 raft_consensus.cc:1261] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761 [term 1 FOLLOWER]: Refusing update from remote peer 90a5681b349d4313a8a0ea9bec4d87fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:56:40.791467 15260 consensus_queue.cc:1035] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:56:40.791702 14870 consensus_queue.cc:1035] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:56:40.800338 15268 mvcc.cc:205] Tried to move back new op lower bound from 7315365072031432704 to 7315365071901036544. Current Snapshot: MvccSnapshot[applied={T|T < 7315365072031432704}]
I20260806 00:56:40.806557 14603 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:56:40.806742 14603 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:56:40.809423 14603 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: Sending DeleteTablet for 3 replicas of tablet 85da5d55e09a4a82a37b90f36ab00054
I20260806 00:56:40.810161 15140 tablet_service.cc:1514] Processing DeleteTablet for tablet 85da5d55e09a4a82a37b90f36ab00054 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:56:40 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:48080
I20260806 00:56:40.810357 14962 tablet_service.cc:1514] Processing DeleteTablet for tablet 85da5d55e09a4a82a37b90f36ab00054 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:56:40 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:41780
I20260806 00:56:40.810654 14771 tablet_service.cc:1514] Processing DeleteTablet for tablet 85da5d55e09a4a82a37b90f36ab00054 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:56:40 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:60182
I20260806 00:56:40.810743 15279 tablet_replica.cc:330] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761: stopping tablet replica
I20260806 00:56:40.810974 15282 tablet_replica.cc:330] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd: stopping tablet replica
I20260806 00:56:40.811092 15282 raft_consensus.cc:2229] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:56:40.811090 15279 raft_consensus.cc:2229] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:56:40.811189 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 14658
I20260806 00:56:40.811379 15282 raft_consensus.cc:2258] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:56:40.811419 15279 raft_consensus.cc:2258] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:56:40.814159 15279 ts_tablet_manager.cc:1905] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:56:40.814183 15280 tablet_replica.cc:330] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68: stopping tablet replica
I20260806 00:56:40.814496 15280 raft_consensus.cc:2229] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:56:40.814836 15280 raft_consensus.cc:2258] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:56:40.817950 15279 ts_tablet_manager.cc:1918] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:56:40.818090 15279 log.cc:1196] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/85da5d55e09a4a82a37b90f36ab00054
I20260806 00:56:40.818552 15279 ts_tablet_manager.cc:1939] T 85da5d55e09a4a82a37b90f36ab00054 P 0ab771c8e62b4bfd83206f3ee3166761: Deleting consensus metadata
I20260806 00:56:40.818853 15280 ts_tablet_manager.cc:1905] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:56:40.820017 14585 catalog_manager.cc:4841] TS 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175): tablet 85da5d55e09a4a82a37b90f36ab00054 (table test-table [id=8abb4cb99f9341fdaa312af5eb4e3d91]) successfully deleted
I20260806 00:56:40.821637 15280 ts_tablet_manager.cc:1918] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:56:40.821730 15280 log.cc:1196] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/85da5d55e09a4a82a37b90f36ab00054
I20260806 00:56:40.822130 15280 ts_tablet_manager.cc:1939] T 85da5d55e09a4a82a37b90f36ab00054 P ace5781cc9314ccf8d24279b51e2cf68: Deleting consensus metadata
I20260806 00:56:40.823468 14582 catalog_manager.cc:4841] TS ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549): tablet 85da5d55e09a4a82a37b90f36ab00054 (table test-table [id=8abb4cb99f9341fdaa312af5eb4e3d91]) successfully deleted
W20260806 00:56:40.824832 14582 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:35109 (error 108)
I20260806 00:56:40.825794 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 14867
I20260806 00:56:40.826187 15247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:40.825776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:35109 (local address 127.0.0.1:60206)
0806 00:56:40.825958 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:40.826035 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:35109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 00:56:40.828452 14582 catalog_manager.cc:4584] TS 90a5681b349d4313a8a0ea9bec4d87fd (127.27.77.193:35109): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 85da5d55e09a4a82a37b90f36ab00054: Network error: Client connection negotiation failed: client connection to 127.27.77.193:35109: connect: Connection refused (error 111)
I20260806 00:56:40.835973 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 15050
I20260806 00:56:40.846272 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 14551
I20260806 00:56:40.857779 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:41297
--webserver_interface=127.27.77.254
--webserver_port=42483
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:39101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:41297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:40.972820 15284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:40.973117 15284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:40.973217 15284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:40.977299 15284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:56:40.977361 15284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:40.977381 15284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:40.977397 15284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:56:40.977412 15284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:56:40.982816 15284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:39101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:41297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:41297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=42483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:40.984046 15284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:40.985111 15284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:40.994098 15290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:40.994163 15292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:40.994252 15284 server_base.cc:1034] running on GCE node
W20260806 00:56:40.994454 15294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:40.995085 15284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:40.996315 15284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:40.997512 15284 hybrid_clock.cc:648] HybridClock initialized: now 1785977800997499 us; error 27 us; skew 500 ppm
Aug 06 00:56:40 dist-test-slave-sb2z krb5kdc[13799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977800, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:41.000619 15284 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:56:41.001637 15284 webserver.cc:460] Webserver started at http://127.27.77.254:42483/ using document root <none> and password file <none>
I20260806 00:56:41.002180 15284 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:41.002246 15284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:41.006235 15284 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:56:41.008366 15301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:41.009775 15284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:56:41.009940 15284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "74621d8113284ab7abe2a8be5ab6ca41"
format_stamp: "Formatted at 2026-08-06 00:56:38 on dist-test-slave-sb2z"
server_key: "94af1c229611d3d503461233998305a7"
server_key_iv: "ff7fb31f8ca99ba1f402d23284928e3c"
server_key_version: "encryptionkey@0"
I20260806 00:56:41.010504 15284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:41.043859 15284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:41.047405 15284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:41.047600 15284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:41.055063 15284 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:41297
I20260806 00:56:41.055375 15380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:41297 every 8 connection(s)
I20260806 00:56:41.056300 15284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:56:41.060703 15381 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:56:41.062431 15381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: Bootstrap starting.
I20260806 00:56:41.065574 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 15284
I20260806 00:56:41.066054 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:35109
--local_ip_for_outbound_sockets=127.27.77.193
--tserver_master_addrs=127.27.77.254:41297
--webserver_port=35169
--webserver_interface=127.27.77.193
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:39101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:56:41.081816 15381 log.cc:824] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:41.093652 15381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:56:41.094077 15381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41: Bootstrap complete.
I20260806 00:56:41.098383 15381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } }
I20260806 00:56:41.098812 15381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74621d8113284ab7abe2a8be5ab6ca41, State: Initialized, Role: FOLLOWER
I20260806 00:56:41.099314 15381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } }
I20260806 00:56:41.099481 15381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:56:41.099596 15381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:56:41.099737 15381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 2 FOLLOWER]: Advancing to term 3
I20260806 00:56:41.101670 15381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } }
I20260806 00:56:41.102005 15381 leader_election.cc:302] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74621d8113284ab7abe2a8be5ab6ca41; no voters:
I20260806 00:56:41.102355 15381 leader_election.cc:288] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 00:56:41.102451 15385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 00:56:41.102653 15385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [term 3 LEADER]: Becoming Leader. State: Replica: 74621d8113284ab7abe2a8be5ab6ca41, State: Running, Role: LEADER
I20260806 00:56:41.102963 15385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } }
I20260806 00:56:41.103672 15381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:56:41.103947 15385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74621d8113284ab7abe2a8be5ab6ca41. Latest consensus state: current_term: 3 leader_uuid: "74621d8113284ab7abe2a8be5ab6ca41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } } }
I20260806 00:56:41.104072 15385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:41.104490 15387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "74621d8113284ab7abe2a8be5ab6ca41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74621d8113284ab7abe2a8be5ab6ca41" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 41297 } } }
I20260806 00:56:41.104677 15387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74621d8113284ab7abe2a8be5ab6ca41 [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:41.105840 15390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:56:41.111205 15390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:56:41.111922 15390 catalog_manager.cc:1259] Loaded cluster ID: ff25458b51a84697a11be8b4b137840f
I20260806 00:56:41.111995 15390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 00:56:41.113127 15390 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f31ae6169ad google::LogMessage::Fail() at ??:0
@ 0x7f31ae61988a google::LogMessage::SendToLog() at ??:0
@ 0x7f31ae6164de google::LogMessage::Flush() at ??:0
@ 0x7f31ae617a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f31b8f55fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f31b8f56d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f31b8f53579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f31b8f97640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f31b08683d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f31b0868c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f31b086a3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f31b0859512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f31b1e9d6db start_thread at ??:0
@ 0x7f31ad11461f clone at ??:0
W20260806 00:56:41.203820 15383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:41.204071 15383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:41.204145 15383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:41.208226 15383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:41.208282 15383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:41.208369 15383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:56:41.213814 15383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:39101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:35109
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=35169
--tserver_master_addrs=127.27.77.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:41.214926 15383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:41.216056 15383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:41.226171 15436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:41.226166 15435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:41.226379 15439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:41.226505 15383 server_base.cc:1034] running on GCE node
I20260806 00:56:41.227129 15383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:41.227905 15383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:41.229135 15383 hybrid_clock.cc:648] HybridClock initialized: now 1785977801229106 us; error 48 us; skew 500 ppm
Aug 06 00:56:41 dist-test-slave-sb2z krb5kdc[13799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977801, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:41.232090 15383 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:56:41.233145 15383 webserver.cc:460] Webserver started at http://127.27.77.193:35169/ using document root <none> and password file <none>
I20260806 00:56:41.233765 15383 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:41.233830 15383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:41.238068 15383 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:56:41.240356 15448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:41.241715 15383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:56:41.241814 15383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "90a5681b349d4313a8a0ea9bec4d87fd"
format_stamp: "Formatted at 2026-08-06 00:56:38 on dist-test-slave-sb2z"
server_key: "34ed7a9f34fac7e3a891f068f5c0771c"
server_key_iv: "7a22f727c9686fa7af94a57ea5720d6e"
server_key_version: "encryptionkey@0"
I20260806 00:56:41.242403 15383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:41.260838 15383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:41.264357 15383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:41.264565 15383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:41.265184 15383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:41.266348 15455 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 00:56:41.272271 15383 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 00:56:41.272357 15383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260806 00:56:41.272445 15383 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 00:56:41.274191 15383 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 00:56:41.274262 15383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 00:56:41.274609 15455 tablet_bootstrap.cc:492] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd: Bootstrap starting.
I20260806 00:56:41.285944 15383 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:35109
I20260806 00:56:41.286125 15609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:35109 every 8 connection(s)
I20260806 00:56:41.287185 15383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:56:41.294416 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 15383
I20260806 00:56:41.294945 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:36549
--local_ip_for_outbound_sockets=127.27.77.194
--tserver_master_addrs=127.27.77.254:41297
--webserver_port=45961
--webserver_interface=127.27.77.194
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:39101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:56:41.301965 15455 log.cc:824] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:41.309510 15455 tablet_bootstrap.cc:492] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:56:41.309926 15455 tablet_bootstrap.cc:492] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd: Bootstrap complete.
I20260806 00:56:41.311213 15455 ts_tablet_manager.cc:1397] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd: Time spent bootstrapping tablet: real 0.037s user 0.023s sys 0.008s
I20260806 00:56:41.313838 15455 raft_consensus.cc:348] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:41.314287 15455 raft_consensus.cc:729] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90a5681b349d4313a8a0ea9bec4d87fd, State: Initialized, Role: FOLLOWER
I20260806 00:56:41.314785 15455 consensus_queue.cc:260] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:41.315834 15455 ts_tablet_manager.cc:1428] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
W20260806 00:56:41.395190 15619 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:56:41.289347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:45571)
0806 00:56:41.289845 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:41.289899 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:41.290743 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:41.394400 (+103657us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":144,"threads_started":1}
W20260806 00:56:41.395803 15611 heartbeater.cc:644] Failed to heartbeat to 127.27.77.254:41297 (0 consecutive failures): Network error: Failed to ping master at 127.27.77.254:41297: Client connection negotiation failed: client connection to 127.27.77.254:41297: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 00:56:41.396633 15619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:41.396359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:43749)
0806 00:56:41.396497 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:41.396574 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 00:56:41.397361 15619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:41.397159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:43129)
0806 00:56:41.397267 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:41.397311 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 00:56:41.397624 15611 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:56:41.433145 15620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:41.433379 15620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:41.433466 15620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:41.437592 15620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:41.437650 15620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:41.437744 15620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:56:41.442986 15620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:39101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:36549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=45961
--tserver_master_addrs=127.27.77.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:41.444181 15620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:41.445220 15620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:41.455422 15628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:41.455428 15627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:41.455422 15630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:41.455773 15620 server_base.cc:1034] running on GCE node
I20260806 00:56:41.456241 15620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:41.456926 15620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:41.458150 15620 hybrid_clock.cc:648] HybridClock initialized: now 1785977801458111 us; error 51 us; skew 500 ppm
Aug 06 00:56:41 dist-test-slave-sb2z krb5kdc[13799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977801, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:41.461210 15620 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:56:41.462127 15620 webserver.cc:460] Webserver started at http://127.27.77.194:45961/ using document root <none> and password file <none>
I20260806 00:56:41.462733 15620 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:41.462833 15620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:41.466889 15620 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:56:41.469103 15638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:41.470492 15620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 00:56:41.470647 15620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "ace5781cc9314ccf8d24279b51e2cf68"
format_stamp: "Formatted at 2026-08-06 00:56:38 on dist-test-slave-sb2z"
server_key: "8322a617b3a5648aec8b13549e6c80a0"
server_key_iv: "fa84f641a1e29ccecd6965f17e0a7c5a"
server_key_version: "encryptionkey@0"
I20260806 00:56:41.471212 15620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:41.489547 15620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:41.492760 15620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:41.492995 15620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:41.493634 15620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:41.494494 15620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:41.494571 15620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:41.494647 15620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:41.494695 15620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:41.504199 15620 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:36549
I20260806 00:56:41.504197 15802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:36549 every 8 connection(s)
I20260806 00:56:41.505236 15620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:56:41.508901 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:41.507581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:33391)
0806 00:56:41.508085 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:41.508201 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":117,"threads_started":1}
W20260806 00:56:41.509361 15803 heartbeater.cc:644] Failed to heartbeat to 127.27.77.254:41297 (0 consecutive failures): Network error: Failed to ping master at 127.27.77.254:41297: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
I20260806 00:56:41.509991 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:41.509780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:32947)
0806 00:56:41.509885 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:41.509933 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 00:56:41.510679 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:41.510489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:58643)
0806 00:56:41.510581 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:41.510623 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 00:56:41.510936 15803 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 00:56:41.515591 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 15620
I20260806 00:56:41.516022 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:44175
--local_ip_for_outbound_sockets=127.27.77.195
--tserver_master_addrs=127.27.77.254:41297
--webserver_port=43673
--webserver_interface=127.27.77.195
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:39101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:41.632290 15808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:41.632552 15808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:41.632596 15808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:41.636938 15808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:41.637010 15808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:41.637069 15808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:56:41.642444 15808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:39101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:44175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=43673
--tserver_master_addrs=127.27.77.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:41.643631 15808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:41.644704 15808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:41.655303 15815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:41.655397 15808 server_base.cc:1034] running on GCE node
W20260806 00:56:41.655284 15817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:41.655284 15820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:41.656116 15808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:41.656800 15808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:41.657998 15808 hybrid_clock.cc:648] HybridClock initialized: now 1785977801657965 us; error 46 us; skew 500 ppm
Aug 06 00:56:41 dist-test-slave-sb2z krb5kdc[13799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977801, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:41.661020 15808 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:56:41.661988 15808 webserver.cc:460] Webserver started at http://127.27.77.195:43673/ using document root <none> and password file <none>
I20260806 00:56:41.662575 15808 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:41.662671 15808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:41.666677 15808 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:56:41.668859 15831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:41.670151 15808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:56:41.670313 15808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "0ab771c8e62b4bfd83206f3ee3166761"
format_stamp: "Formatted at 2026-08-06 00:56:39 on dist-test-slave-sb2z"
server_key: "cf482e91ee8011546b9132c1649717e3"
server_key_iv: "faea17daf0cabeaa4c737ee384392108"
server_key_version: "encryptionkey@0"
I20260806 00:56:41.670867 15808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:41.700682 15808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:41.703859 15808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:41.704090 15808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:41.704771 15808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:41.705659 15808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:41.705734 15808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:41.705809 15808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:41.705859 15808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:41.715610 15808 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:44175
I20260806 00:56:41.715623 15981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:44175 every 8 connection(s)
I20260806 00:56:41.716652 15808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:56:41.720647 15984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:41.718899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:59399)
0806 00:56:41.719395 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:41.719930 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":111,"threads_started":1}
W20260806 00:56:41.721100 15982 heartbeater.cc:644] Failed to heartbeat to 127.27.77.254:41297 (0 consecutive failures): Network error: Failed to ping master at 127.27.77.254:41297: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
I20260806 00:56:41.721741 15984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:41.721511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:55969)
0806 00:56:41.721631 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:41.721686 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 00:56:41.722422 15984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:41.722189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:40481)
0806 00:56:41.722312 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:41.722362 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 00:56:41.722638 15982 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 00:56:41.724349 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 15808
I20260806 00:56:42.398718 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:42.398227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:54269)
0806 00:56:42.398539 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:42.398599 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":77,"threads_started":1}
I20260806 00:56:42.512275 15988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:42.511658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:56673)
0806 00:56:42.512014 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:42.512095 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":75,"threads_started":1}
I20260806 00:56:42.723798 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:42.723308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:42231)
0806 00:56:42.723640 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:42.723694 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":107,"threads_started":1}
I20260806 00:56:42.906925 15991 raft_consensus.cc:482] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:56:42.907163 15991 raft_consensus.cc:504] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:42.908334 15991 leader_election.cc:288] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175), ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549)
Aug 06 00:56:42 dist-test-slave-sb2z krb5kdc[13799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977801, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.194@KRBTEST.COM
Aug 06 00:56:42 dist-test-slave-sb2z krb5kdc[13799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977801, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.195@KRBTEST.COM
I20260806 00:56:42.923473 15992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:42.908643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:36549 (local address 127.27.77.193:36359)
0806 00:56:42.909066 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:42.909081 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:42.909217 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:42.910911 (+ 1694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:42.910918 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:42.911480 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:42.912017 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:42.912035 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:42.913327 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:42.913335 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:42.914302 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:42.914312 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:42.915037 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:42.915040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:42.915211 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:42.916023 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:42.916041 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:42.917715 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:42.920338 (+ 2623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:42.920343 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:42.920355 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:42.920757 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:42.921193 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:42.921196 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:42.921199 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:42.921355 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:42.922218 (+ 863us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:42.922226 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:42.922626 (+ 400us) client_negotiation.cc:769] Sending connection context
0806 00:56:42.922856 (+ 230us) client_negotiation.cc:240] Negotiation successful
0806 00:56:42.923297 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":264,"spinlock_wait_cycles":15744,"thread_start_us":80,"threads_started":1}
I20260806 00:56:42.923559 15994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:42.908782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:36359 (local address 127.27.77.194:36549)
0806 00:56:42.909218 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:42.909246 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:42.909296 (+ 50us) server_negotiation.cc:407] Connection header received
0806 00:56:42.910130 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:42.910139 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:42.910559 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:42.910862 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:42.912195 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:42.913181 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:42.914435 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:42.914907 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:42.915107 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:42.917956 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:42.917961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:42.917973 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:42.918000 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:42.920196 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:42.920889 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:42.920898 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:42.920904 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:42.921079 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:42.921489 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:42.921491 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:42.921493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:42.922081 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:42.922235 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:42.923028 (+ 793us) server_negotiation.cc:299] Negotiation successful
0806 00:56:42.923340 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":60,"threads_started":1}
I20260806 00:56:42.923563 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:42.908806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:45015 (local address 127.27.77.195:44175)
0806 00:56:42.909264 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:42.909292 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:42.909335 (+ 43us) server_negotiation.cc:407] Connection header received
0806 00:56:42.910276 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:42.910287 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:42.910623 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:42.910932 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:42.912221 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:42.913263 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:42.914454 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:42.914941 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:42.915131 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:42.918307 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:42.918317 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:42.918336 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:42.918376 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:42.920581 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:42.921112 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:42.921119 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:42.921128 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:42.921320 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:42.921623 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:42.921626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:42.921628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:42.922081 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:42.922268 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:42.923028 (+ 760us) server_negotiation.cc:299] Negotiation successful
0806 00:56:42.923340 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":104,"threads_started":1}
I20260806 00:56:42.923475 15993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:42.908630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:44175 (local address 127.27.77.193:45015)
0806 00:56:42.909036 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:42.909057 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:42.909215 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:42.910921 (+ 1706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:42.910925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:42.911487 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:42.912019 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:42.912034 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:42.913407 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:42.913413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:42.914300 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:42.914311 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:42.915081 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:42.915085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:42.915214 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:42.916018 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:42.916037 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:42.918079 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:42.920696 (+ 2617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:42.920702 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:42.920705 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:42.920985 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:42.921424 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:42.921429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:42.921432 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:42.921528 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:42.922231 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:42.922233 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:42.922626 (+ 393us) client_negotiation.cc:769] Sending connection context
0806 00:56:42.922856 (+ 230us) client_negotiation.cc:240] Negotiation successful
0806 00:56:42.923244 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":233,"spinlock_wait_cycles":56064,"thread_start_us":87,"threads_started":1}
W20260806 00:56:42.926241 15452 leader_election.cc:341] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175): Not found: Tablet not found: 85da5d55e09a4a82a37b90f36ab00054
W20260806 00:56:42.926443 15450 leader_election.cc:341] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549): Not found: Tablet not found: 85da5d55e09a4a82a37b90f36ab00054
I20260806 00:56:42.926543 15450 leader_election.cc:302] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90a5681b349d4313a8a0ea9bec4d87fd; no voters: 0ab771c8e62b4bfd83206f3ee3166761, ace5781cc9314ccf8d24279b51e2cf68
I20260806 00:56:42.926877 15991 raft_consensus.cc:2726] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:56:43.399956 15992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:43.399632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:38883)
0806 00:56:43.399789 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:43.399854 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 00:56:43.513551 15996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:43.513132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:51905)
0806 00:56:43.513393 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:43.513454 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":80,"threads_started":1}
I20260806 00:56:43.725030 15998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:43.724590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:34459)
0806 00:56:43.724873 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:43.724937 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":86,"threads_started":1}
I20260806 00:56:44.401217 15999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:44.400741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:55077)
0806 00:56:44.401031 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:44.401090 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":74,"threads_started":1}
I20260806 00:56:44.511771 16000 raft_consensus.cc:482] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:56:44.511920 16000 raft_consensus.cc:504] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:44.512492 16000 leader_election.cc:288] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175), ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549)
W20260806 00:56:44.513456 15452 leader_election.cc:341] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175): Not found: Tablet not found: 85da5d55e09a4a82a37b90f36ab00054
W20260806 00:56:44.513583 15450 leader_election.cc:341] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549): Not found: Tablet not found: 85da5d55e09a4a82a37b90f36ab00054
I20260806 00:56:44.513716 15450 leader_election.cc:302] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90a5681b349d4313a8a0ea9bec4d87fd; no voters: 0ab771c8e62b4bfd83206f3ee3166761, ace5781cc9314ccf8d24279b51e2cf68
I20260806 00:56:44.513908 16000 raft_consensus.cc:2726] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:56:44.514729 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:44.514281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:50161)
0806 00:56:44.514557 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:44.514619 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":77,"threads_started":1}
I20260806 00:56:44.726276 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:44.725854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:45923)
0806 00:56:44.726115 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:44.726169 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":93,"threads_started":1}
I20260806 00:56:45.402611 16004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:45.402089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:46495)
0806 00:56:45.402435 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:45.402505 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":152,"threads_started":1}
I20260806 00:56:45.516525 16006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:45.515935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:43627)
0806 00:56:45.516339 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:45.516410 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":108,"threads_started":1}
I20260806 00:56:45.727425 16007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:45.727002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:41957)
0806 00:56:45.727276 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:45.727330 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":96,"threads_started":1}
I20260806 00:56:46.404222 16009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:46.403714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:51687)
0806 00:56:46.404055 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:46.404115 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":83,"threads_started":1}
I20260806 00:56:46.505383 16011 raft_consensus.cc:482] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:56:46.505553 16011 raft_consensus.cc:504] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:46.506043 16011 leader_election.cc:288] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175), ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549)
W20260806 00:56:46.506978 15450 leader_election.cc:341] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549): Not found: Tablet not found: 85da5d55e09a4a82a37b90f36ab00054
W20260806 00:56:46.507098 15452 leader_election.cc:341] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175): Not found: Tablet not found: 85da5d55e09a4a82a37b90f36ab00054
I20260806 00:56:46.507213 15452 leader_election.cc:302] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90a5681b349d4313a8a0ea9bec4d87fd; no voters: 0ab771c8e62b4bfd83206f3ee3166761, ace5781cc9314ccf8d24279b51e2cf68
I20260806 00:56:46.507395 16011 raft_consensus.cc:2726] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:56:46.517840 16012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:46.517402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:43573)
0806 00:56:46.517687 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:46.517747 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":94,"threads_started":1}
I20260806 00:56:46.728767 16013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:46.728274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:60489)
0806 00:56:46.728593 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:46.728668 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":91,"threads_started":1}
I20260806 00:56:47.405582 16014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:47.405102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:38379)
0806 00:56:47.405410 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:47.405480 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":84,"threads_started":1}
I20260806 00:56:47.519146 16016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:47.518624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:48893)
0806 00:56:47.518975 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:47.519049 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":110,"threads_started":1}
I20260806 00:56:47.730049 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:47.729567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:51189)
0806 00:56:47.729868 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:47.729937 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":86,"threads_started":1}
I20260806 00:56:48.406888 16019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:48.406409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:55683)
0806 00:56:48.406716 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:48.406783 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":83,"threads_started":1}
I20260806 00:56:48.520586 16020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:48.520067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:35333)
0806 00:56:48.520424 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:48.520477 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":90,"threads_started":1}
I20260806 00:56:48.537962 16021 raft_consensus.cc:482] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:56:48.538169 16021 raft_consensus.cc:504] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:48.538666 16021 leader_election.cc:288] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175), ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549)
W20260806 00:56:48.539759 15452 leader_election.cc:341] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175): Not found: Tablet not found: 85da5d55e09a4a82a37b90f36ab00054
W20260806 00:56:48.539872 15450 leader_election.cc:341] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549): Not found: Tablet not found: 85da5d55e09a4a82a37b90f36ab00054
I20260806 00:56:48.539990 15450 leader_election.cc:302] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90a5681b349d4313a8a0ea9bec4d87fd; no voters: 0ab771c8e62b4bfd83206f3ee3166761, ace5781cc9314ccf8d24279b51e2cf68
I20260806 00:56:48.540192 16021 raft_consensus.cc:2726] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:56:48.731325 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:48.730853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:40073)
0806 00:56:48.731169 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:48.731226 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":93,"threads_started":1}
I20260806 00:56:49.408373 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:49.407934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:55169)
0806 00:56:49.408211 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:49.408270 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":87,"threads_started":1}
I20260806 00:56:49.522025 16025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:49.521516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:57679)
0806 00:56:49.521849 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:49.521920 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":97,"threads_started":1}
I20260806 00:56:49.732751 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:49.732195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:46247)
0806 00:56:49.732497 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:49.732569 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":83,"threads_started":1}
I20260806 00:56:50.409708 16028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:50.409227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:55321)
0806 00:56:50.409541 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:50.409604 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":97,"threads_started":1}
I20260806 00:56:50.523416 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:50.522904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:40953)
0806 00:56:50.523238 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:50.523313 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":108,"threads_started":1}
I20260806 00:56:50.734287 16030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:50.733650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:39925)
0806 00:56:50.734009 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:50.734131 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":87,"threads_started":1}
I20260806 00:56:51.411216 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:51.410730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:40899)
0806 00:56:51.411013 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:51.411113 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":82,"threads_started":1}
I20260806 00:56:51.524788 16033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:51.524330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:51795)
0806 00:56:51.524613 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:51.524685 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":84,"threads_started":1}
I20260806 00:56:51.735762 16034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:51.735259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:58169)
0806 00:56:51.735589 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:51.735653 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":95,"threads_started":1}
I20260806 00:56:52.412511 16035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:52.412045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:59011)
0806 00:56:52.412344 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:52.412408 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":80,"threads_started":1}
I20260806 00:56:52.526240 16042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:52.525711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:39905)
0806 00:56:52.526068 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:52.526134 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":86,"threads_started":1}
I20260806 00:56:52.737296 16043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:52.736759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:53047)
0806 00:56:52.737111 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:52.737183 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":97,"threads_started":1}
I20260806 00:56:53.413935 16044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:53.413399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:36563)
0806 00:56:53.413777 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:53.413833 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":155,"threads_started":1}
I20260806 00:56:53.527773 16045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:53.527179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:35871)
0806 00:56:53.527478 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:53.527634 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":89,"threads_started":1}
I20260806 00:56:53.738719 16047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:53.738196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:53071)
0806 00:56:53.738559 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:53.738621 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":101,"threads_started":1}
I20260806 00:56:54.415269 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:54.414797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:50983)
0806 00:56:54.415107 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:54.415169 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":87,"threads_started":1}
I20260806 00:56:54.529211 16049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:54.528712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:34177)
0806 00:56:54.529030 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:54.529106 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":90,"threads_started":1}
I20260806 00:56:54.740231 16050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:54.739778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:38533)
0806 00:56:54.740062 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:54.740127 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":82,"threads_started":1}
I20260806 00:56:55.052703 16051 raft_consensus.cc:482] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:56:55.052850 16051 raft_consensus.cc:504] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab771c8e62b4bfd83206f3ee3166761" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44175 } } peers { permanent_uuid: "ace5781cc9314ccf8d24279b51e2cf68" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36549 } } peers { permanent_uuid: "90a5681b349d4313a8a0ea9bec4d87fd" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 35109 } }
I20260806 00:56:55.053273 16051 leader_election.cc:288] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175), ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549)
W20260806 00:56:55.054268 15452 leader_election.cc:341] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ab771c8e62b4bfd83206f3ee3166761 (127.27.77.195:44175): Not found: Tablet not found: 85da5d55e09a4a82a37b90f36ab00054
W20260806 00:56:55.054407 15450 leader_election.cc:341] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ace5781cc9314ccf8d24279b51e2cf68 (127.27.77.194:36549): Not found: Tablet not found: 85da5d55e09a4a82a37b90f36ab00054
I20260806 00:56:55.054500 15450 leader_election.cc:302] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90a5681b349d4313a8a0ea9bec4d87fd; no voters: 0ab771c8e62b4bfd83206f3ee3166761, ace5781cc9314ccf8d24279b51e2cf68
I20260806 00:56:55.054685 16051 raft_consensus.cc:2726] T 85da5d55e09a4a82a37b90f36ab00054 P 90a5681b349d4313a8a0ea9bec4d87fd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:56:55.416647 16053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:55.416164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:47601)
0806 00:56:55.416474 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:55.416542 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":101,"threads_started":1}
I20260806 00:56:55.530664 16054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:55.530183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:40435)
0806 00:56:55.530497 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:55.530560 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":84,"threads_started":1}
I20260806 00:56:55.741503 16056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:55.741020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:41091)
0806 00:56:55.741319 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:55.741384 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":89,"threads_started":1}
I20260806 00:56:56.418123 16057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:56.417687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.193:48313)
0806 00:56:56.417942 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:56.418011 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":76,"threads_started":1}
I20260806 00:56:56.532258 16058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:56.531809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.194:35719)
0806 00:56:56.532089 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:56.532151 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":78,"threads_started":1}
I20260806 00:56:56.726711 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 15383
I20260806 00:56:56.741868 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 15620
I20260806 00:56:56.747151 16060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:56.746471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:41297 (local address 127.27.77.195:37951)
0806 00:56:56.746956 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:56.747035 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":80,"threads_started":1}
I20260806 00:56:56.753438 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 15808
2026-08-06T00:56:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19897 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:56:56 dist-test-slave-sb2z krb5kdc[16063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:56:56 dist-test-slave-sb2z krb5kdc[16063](info): set up 2 sockets
Aug 06 00:56:56 dist-test-slave-sb2z krb5kdc[16063](info): commencing operation
krb5kdc: starting...
W20260806 00:56:57.959501 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:56:58 dist-test-slave-sb2z krb5kdc[16063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:56:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:56:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:56:58.098681 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38001
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:38001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:58.215472 16084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:58.215838 16084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:58.215917 16084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:58.220012 16084 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 00:56:58.220091 16084 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 00:56:58.220119 16084 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 00:56:58.220144 16084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:56:58.220188 16084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:58.220228 16084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:58.220264 16084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:56:58.220301 16084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:56:58.225778 16084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:38001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:58.226935 16084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:58.228093 16084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:58.238188 16094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:58.238181 16096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:58.238243 16084 server_base.cc:1034] running on GCE node
W20260806 00:56:58.238178 16093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:58.239099 16084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:58.240420 16084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:58.241598 16084 hybrid_clock.cc:648] HybridClock initialized: now 1785977818241561 us; error 61 us; skew 500 ppm
Aug 06 00:56:58 dist-test-slave-sb2z krb5kdc[16063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977818, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:58.244582 16084 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:56:58.245563 16084 webserver.cc:460] Webserver started at http://127.27.77.254:46105/ using document root <none> and password file <none>
I20260806 00:56:58.246222 16084 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:58.246316 16084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:58.246565 16084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:56:58.248910 16084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "6a0f517681684136a90430c795ded7cc"
format_stamp: "Formatted at 2026-08-06 00:56:58 on dist-test-slave-sb2z"
server_key: "2e466672b1492ea07ca2ef943c6068d5"
server_key_iv: "bf57644131e95e3cf9b73799196fa4d8"
server_key_version: "encryptionkey@0"
I20260806 00:56:58.249532 16084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "6a0f517681684136a90430c795ded7cc"
format_stamp: "Formatted at 2026-08-06 00:56:58 on dist-test-slave-sb2z"
server_key: "2e466672b1492ea07ca2ef943c6068d5"
server_key_iv: "bf57644131e95e3cf9b73799196fa4d8"
server_key_version: "encryptionkey@0"
I20260806 00:56:58.253546 16084 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:56:58.256253 16106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:58.258174 16084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:56:58.258352 16084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "6a0f517681684136a90430c795ded7cc"
format_stamp: "Formatted at 2026-08-06 00:56:58 on dist-test-slave-sb2z"
server_key: "2e466672b1492ea07ca2ef943c6068d5"
server_key_iv: "bf57644131e95e3cf9b73799196fa4d8"
server_key_version: "encryptionkey@0"
I20260806 00:56:58.258496 16084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:58.285233 16084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:58.290333 16084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:58.290619 16084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:58.299503 16084 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:38001
I20260806 00:56:58.299873 16178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:38001 every 8 connection(s)
I20260806 00:56:58.300835 16084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:56:58.304358 16179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:58.307267 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 16084
I20260806 00:56:58.307396 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:56:58.307807 27959 external_mini_cluster.cc:1428] Setting key 046c4c589b63048a5688c5be164a42ff
I20260806 00:56:58.311429 16179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Bootstrap starting.
I20260806 00:56:58.314414 16179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:58.315268 16179 log.cc:824] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:58.317732 16179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: No bootstrap required, opened a new log
Aug 06 00:56:58 dist-test-slave-sb2z krb5kdc[16063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:58.322256 16179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } }
I20260806 00:56:58.322593 16179 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:58.322685 16179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a0f517681684136a90430c795ded7cc, State: Initialized, Role: FOLLOWER
I20260806 00:56:58.323242 16179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } }
I20260806 00:56:58.323410 16179 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:56:58.323509 16179 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:56:58.323647 16179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:58.324944 16179 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } }
I20260806 00:56:58.325347 16179 leader_election.cc:302] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a0f517681684136a90430c795ded7cc; no voters:
I20260806 00:56:58.325753 16179 leader_election.cc:288] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:56:58.325845 16184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:56:58.326038 16184 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 1 LEADER]: Becoming Leader. State: Replica: 6a0f517681684136a90430c795ded7cc, State: Running, Role: LEADER
I20260806 00:56:58.326288 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:58.309500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40366 (local address 127.27.77.254:38001)
0806 00:56:58.310073 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:58.310095 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:58.310146 (+ 51us) server_negotiation.cc:407] Connection header received
0806 00:56:58.311039 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:58.311065 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:58.311515 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:58.311946 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:58.312835 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:58.314336 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:58.315492 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:58.316108 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:58.316282 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:58.319195 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:58.319201 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:58.319221 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:58.319253 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:58.322271 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:58.322877 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:58.322883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:58.322889 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:58.323046 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:58.323451 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:58.323458 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:58.323462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:58.324065 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:58.324316 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:58.324667 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 00:56:58.325168 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":152,"threads_started":1}
I20260806 00:56:58.326419 16184 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } }
I20260806 00:56:58.327190 16179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:56:58.328611 16189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a0f517681684136a90430c795ded7cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } } }
I20260806 00:56:58.328634 16186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a0f517681684136a90430c795ded7cc. Latest consensus state: current_term: 1 leader_uuid: "6a0f517681684136a90430c795ded7cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } } }
I20260806 00:56:58.328812 16189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:58.328931 16186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [sys.catalog]: This master's current role is: LEADER
I20260806 00:56:58.330034 16192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:56:58.333252 16192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:56:58.339464 16192 catalog_manager.cc:1347] Generated new cluster ID: c159e7cb46da44bf9f56636be35754f6
I20260806 00:56:58.339609 16192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:56:58.348428 16192 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:56:58.349786 16192 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:56:58.359246 16192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Generated new TSK 0
I20260806 00:56:58.360163 16192 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:56:58.406255 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38001
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:58.550992 16215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:58.551316 16215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:58.551364 16215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:58.555832 16215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:58.555910 16215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:58.555984 16215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:56:58.561729 16215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:58.563089 16215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:58.564318 16215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:58.575094 16223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:58.575189 16222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:58.575068 16226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:58.575408 16215 server_base.cc:1034] running on GCE node
I20260806 00:56:58.575908 16215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:58.576699 16215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:58.577905 16215 hybrid_clock.cc:648] HybridClock initialized: now 1785977818577878 us; error 42 us; skew 500 ppm
Aug 06 00:56:58 dist-test-slave-sb2z krb5kdc[16063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977818, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:58.581166 16215 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:56:58.582208 16215 webserver.cc:460] Webserver started at http://127.27.77.193:43965/ using document root <none> and password file <none>
I20260806 00:56:58.582858 16215 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:58.583012 16215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:58.583271 16215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:56:58.585657 16215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "a185f155050e4e458a419bf1e80b84e6"
format_stamp: "Formatted at 2026-08-06 00:56:58 on dist-test-slave-sb2z"
server_key: "1da3955ee1ba2127471670fb38cdfa3d"
server_key_iv: "a86a44c4dae836089752fc4c615c1e3d"
server_key_version: "encryptionkey@0"
I20260806 00:56:58.586297 16215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "a185f155050e4e458a419bf1e80b84e6"
format_stamp: "Formatted at 2026-08-06 00:56:58 on dist-test-slave-sb2z"
server_key: "1da3955ee1ba2127471670fb38cdfa3d"
server_key_iv: "a86a44c4dae836089752fc4c615c1e3d"
server_key_version: "encryptionkey@0"
I20260806 00:56:58.590581 16215 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:56:58.593490 16234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:58.594861 16215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:56:58.595057 16215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "a185f155050e4e458a419bf1e80b84e6"
format_stamp: "Formatted at 2026-08-06 00:56:58 on dist-test-slave-sb2z"
server_key: "1da3955ee1ba2127471670fb38cdfa3d"
server_key_iv: "a86a44c4dae836089752fc4c615c1e3d"
server_key_version: "encryptionkey@0"
I20260806 00:56:58.595196 16215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:58.611305 16215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:58.614902 16215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:58.615165 16215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:58.615957 16215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:58.616927 16215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:58.617017 16215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:58.617098 16215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:58.617153 16215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:58.627458 16215 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:46279
I20260806 00:56:58.627485 16383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:46279 every 8 connection(s)
I20260806 00:56:58.628659 16215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:56:58.637235 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 16215
I20260806 00:56:58.637408 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:56:58.637740 27959 external_mini_cluster.cc:1428] Setting key 3789bf74cb900b0d6d3c5ad112e7d017
Aug 06 00:56:58 dist-test-slave-sb2z krb5kdc[16063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977818, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:58.644356 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:58.631032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:45279 (local address 127.27.77.254:38001)
0806 00:56:58.631205 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:58.631211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:58.632727 (+ 1516us) server_negotiation.cc:407] Connection header received
0806 00:56:58.633641 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:58.633645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:58.633707 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:58.633816 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:58.635078 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:58.635709 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:58.636960 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:58.637218 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:58.637355 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:58.640395 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:58.640400 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:58.640404 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:58.640437 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:58.642065 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:58.642626 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:58.642630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:58.642632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:58.642724 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:58.643100 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:58.643105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:58.643110 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:58.643305 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:58.643421 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:58.644020 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 00:56:58.644182 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 00:56:58.645253 16386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:58.631606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:45279)
0806 00:56:58.632408 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:58.632461 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:58.633409 (+ 948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:58.633990 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:58.633997 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:58.634382 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:58.634913 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:58.634927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:58.635842 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:58.635846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:58.636799 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:58.636811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:58.637408 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:58.637413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:58.637608 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:58.638396 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:58.638420 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:58.640212 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:58.642199 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:58.642206 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:58.642225 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:58.642510 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:58.642833 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:58.642836 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:58.642839 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:58.642997 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:58.643421 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:58.643430 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:58.643773 (+ 343us) client_negotiation.cc:769] Sending connection context
0806 00:56:58.643979 (+ 206us) client_negotiation.cc:240] Negotiation successful
0806 00:56:58.644364 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":120,"threads_started":1}
I20260806 00:56:58.646862 16384 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38001
I20260806 00:56:58.647174 16384 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:58.647825 16384 heartbeater.cc:511] Master 127.27.77.254:38001 requested a full tablet report, sending...
I20260806 00:56:58.649705 16128 ts_manager.cc:194] Registered new tserver with Master: a185f155050e4e458a419bf1e80b84e6 (127.27.77.193:46279)
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
I20260806 00:56:58.651019 16128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:45279
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:56:58.674770 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38001
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:58.799317 16390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:58.799734 16390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:58.799829 16390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:58.804883 16390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:58.805001 16390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:58.805109 16390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:56:58.810966 16390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:58.812291 16390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:58.813454 16390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:58.823957 16396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:58.823994 16397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:58.823968 16400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:58.824402 16390 server_base.cc:1034] running on GCE node
I20260806 00:56:58.824931 16390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:58.825773 16390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:58.826989 16390 hybrid_clock.cc:648] HybridClock initialized: now 1785977818826926 us; error 76 us; skew 500 ppm
Aug 06 00:56:58 dist-test-slave-sb2z krb5kdc[16063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977818, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:58.830265 16390 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:56:58.831286 16390 webserver.cc:460] Webserver started at http://127.27.77.194:38537/ using document root <none> and password file <none>
I20260806 00:56:58.831945 16390 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:58.832044 16390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:58.832295 16390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:56:58.834551 16390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "0d3adbf2316b4d47a1465ee793d9e85c"
format_stamp: "Formatted at 2026-08-06 00:56:58 on dist-test-slave-sb2z"
server_key: "aa47892fa16621b2df76b0e260d86c76"
server_key_iv: "ab16397e0741c817d618f3918c502ff0"
server_key_version: "encryptionkey@0"
I20260806 00:56:58.835143 16390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "0d3adbf2316b4d47a1465ee793d9e85c"
format_stamp: "Formatted at 2026-08-06 00:56:58 on dist-test-slave-sb2z"
server_key: "aa47892fa16621b2df76b0e260d86c76"
server_key_iv: "ab16397e0741c817d618f3918c502ff0"
server_key_version: "encryptionkey@0"
I20260806 00:56:58.839314 16390 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 00:56:58.842041 16408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:58.843467 16390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:56:58.843664 16390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "0d3adbf2316b4d47a1465ee793d9e85c"
format_stamp: "Formatted at 2026-08-06 00:56:58 on dist-test-slave-sb2z"
server_key: "aa47892fa16621b2df76b0e260d86c76"
server_key_iv: "ab16397e0741c817d618f3918c502ff0"
server_key_version: "encryptionkey@0"
I20260806 00:56:58.843806 16390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:58.868965 16390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:58.872454 16390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:58.872685 16390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:58.873348 16390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:58.874292 16390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:58.874367 16390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:58.874444 16390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:58.874495 16390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:58.885517 16390 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:37553
I20260806 00:56:58.885563 16564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:37553 every 8 connection(s)
I20260806 00:56:58.886538 16390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:56:58.894667 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 16390
I20260806 00:56:58.894783 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:56:58.895119 27959 external_mini_cluster.cc:1428] Setting key 806da3058b4c0b98f55c9ac84af2465c
Aug 06 00:56:58 dist-test-slave-sb2z krb5kdc[16063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977818, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:58.903350 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:58.889012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:36581 (local address 127.27.77.254:38001)
0806 00:56:58.889178 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:58.889182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:58.890505 (+ 1323us) server_negotiation.cc:407] Connection header received
0806 00:56:58.891423 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:58.891426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:58.891479 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:58.891680 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:58.893080 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:58.893693 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:58.894913 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:58.895154 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:58.895284 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:58.898904 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:58.898907 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:58.898909 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:58.898950 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:58.900858 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:58.901477 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:58.901483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:58.901488 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:58.901589 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:58.901965 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:58.901972 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:58.901976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:58.902249 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:58.902359 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:58.902924 (+ 565us) server_negotiation.cc:299] Negotiation successful
0806 00:56:58.903092 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:56:58.904237 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:58.889483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:36581)
0806 00:56:58.890327 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:58.890372 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:58.891197 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:58.891851 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:58.891858 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:58.892316 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:58.892916 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:58.892930 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:58.893823 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:58.893826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:58.894783 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:58.894797 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:58.895457 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:58.895460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:58.895757 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:58.896457 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:58.896481 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:58.898676 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:58.901052 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:58.901058 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:58.901072 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:58.901353 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:58.901694 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:58.901697 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:58.901698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:58.901860 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:58.902368 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:58.902374 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:58.902650 (+ 276us) client_negotiation.cc:769] Sending connection context
0806 00:56:58.902828 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 00:56:58.903203 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":156,"threads_started":1}
I20260806 00:56:58.905563 16565 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38001
I20260806 00:56:58.905814 16565 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:58.906360 16565 heartbeater.cc:511] Master 127.27.77.254:38001 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
I20260806 00:56:58.907629 16128 ts_manager.cc:194] Registered new tserver with Master: 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553)
I20260806 00:56:58.908246 16128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:36581
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:56:58.931423 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38001
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:56:59.060887 16573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:56:59.061421 16573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:56:59.061576 16573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:56:59.066236 16573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:56:59.066331 16573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:56:59.066424 16573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:56:59.071985 16573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:38001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:56:59.073196 16573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:56:59.074309 16573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:56:59.084688 16581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:59.084714 16583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:56:59.084707 16578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:56:59.085045 16573 server_base.cc:1034] running on GCE node
I20260806 00:56:59.085469 16573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:56:59.086227 16573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:56:59.087420 16573 hybrid_clock.cc:648] HybridClock initialized: now 1785977819087391 us; error 41 us; skew 500 ppm
Aug 06 00:56:59 dist-test-slave-sb2z krb5kdc[16063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977819, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:56:59.090533 16573 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:56:59.091759 16573 webserver.cc:460] Webserver started at http://127.27.77.195:43301/ using document root <none> and password file <none>
I20260806 00:56:59.092386 16573 fs_manager.cc:359] Metadata directory not provided
I20260806 00:56:59.092450 16573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:56:59.092631 16573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:56:59.094897 16573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c"
format_stamp: "Formatted at 2026-08-06 00:56:59 on dist-test-slave-sb2z"
server_key: "49c3870aa3e743f9166b3c1937e68a1d"
server_key_iv: "b99df649d2eaecc14ddbf37c26dcfe83"
server_key_version: "encryptionkey@0"
I20260806 00:56:59.095451 16573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c"
format_stamp: "Formatted at 2026-08-06 00:56:59 on dist-test-slave-sb2z"
server_key: "49c3870aa3e743f9166b3c1937e68a1d"
server_key_iv: "b99df649d2eaecc14ddbf37c26dcfe83"
server_key_version: "encryptionkey@0"
I20260806 00:56:59.099723 16573 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:56:59.102492 16593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:59.103875 16573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:56:59.103982 16573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c"
format_stamp: "Formatted at 2026-08-06 00:56:59 on dist-test-slave-sb2z"
server_key: "49c3870aa3e743f9166b3c1937e68a1d"
server_key_iv: "b99df649d2eaecc14ddbf37c26dcfe83"
server_key_version: "encryptionkey@0"
I20260806 00:56:59.104118 16573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:56:59.142098 16573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:56:59.145694 16573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:56:59.145870 16573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:56:59.146567 16573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:56:59.147454 16573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:56:59.147558 16573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:59.147634 16573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:56:59.147686 16573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:56:59.157258 16573 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:36079
I20260806 00:56:59.157302 16744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:36079 every 8 connection(s)
I20260806 00:56:59.158222 16573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:56:59.161082 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 16573
I20260806 00:56:59.161234 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:56:59.161607 27959 external_mini_cluster.cc:1428] Setting key 63e9ad2089cd69d33c4116331dcca037
Aug 06 00:56:59 dist-test-slave-sb2z krb5kdc[16063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977819, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:56:59.173192 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:59.160214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:33993 (local address 127.27.77.254:38001)
0806 00:56:59.160401 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:59.160405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:59.161700 (+ 1295us) server_negotiation.cc:407] Connection header received
0806 00:56:59.162802 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:59.162805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:59.162851 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:59.162974 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:59.164509 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.165153 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.166444 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.166668 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.166754 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:59.169463 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:59.169466 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:59.169468 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:59.169493 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:59.171020 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:59.171673 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:59.171676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:59.171677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:59.171760 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:59.172134 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:59.172136 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:59.172137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:59.172321 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:59.172398 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:59.172955 (+ 557us) server_negotiation.cc:299] Negotiation successful
0806 00:56:59.173066 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 00:56:59.174053 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:59.160601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:33993)
0806 00:56:59.161503 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:59.161564 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:59.162563 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:59.163177 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:59.163188 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:59.163732 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:56:59.164370 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:59.164385 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.165286 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.165289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:59.166312 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:59.166322 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.166814 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.166817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:59.167014 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:59.167722 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:56:59.167740 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:56:59.169294 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:56:59.171148 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:59.171154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:59.171167 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:59.171453 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:59.171859 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:56:59.171862 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:56:59.171864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:56:59.172039 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:56:59.172412 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:56:59.172417 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:56:59.172712 (+ 295us) client_negotiation.cc:769] Sending connection context
0806 00:56:59.172868 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 00:56:59.173146 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":126,"threads_started":1}
I20260806 00:56:59.175621 16746 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38001
I20260806 00:56:59.175875 16746 heartbeater.cc:464] Registering TS with master...
I20260806 00:56:59.176417 16746 heartbeater.cc:511] Master 127.27.77.254:38001 requested a full tablet report, sending...
I20260806 00:56:59.177496 16126 ts_manager.cc:194] Registered new tserver with Master: ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079)
I20260806 00:56:59.178167 16126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:33993
I20260806 00:56:59.188577 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:56:59.198838 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:59.190801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40368 (local address 127.27.77.254:38001)
0806 00:56:59.190946 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:59.190950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:59.191086 (+ 136us) server_negotiation.cc:407] Connection header received
0806 00:56:59.191156 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:59.191158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:59.191202 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:59.191274 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:56:59.192058 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.192626 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.193466 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.193701 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.193786 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:59.195356 (+ 1570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:56:59.195359 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:56:59.195362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:56:59.195387 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:56:59.197030 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:59.197478 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:59.197481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:59.197482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:59.197584 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:56:59.197898 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:56:59.197900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:56:59.197901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:56:59.198137 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:56:59.198244 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:59.198576 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 00:56:59.198703 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:56:59.202710 16126 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:56:59.206143 16126 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:56:59.221697 16764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:59.215411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:46279 (local address 127.0.0.1:45646)
0806 00:56:59.215968 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:59.216018 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:59.216164 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:59.216998 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:59.217004 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:59.217029 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:59.217202 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:59.217212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.218499 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.218504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:59.219902 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:59.219914 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.221069 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.221073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:59.221196 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:59.221283 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 00:56:59.221383 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 00:56:59.221508 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":64,"spinlock_wait_cycles":4864,"thread_start_us":132,"threads_started":1}
I20260806 00:56:59.221740 16767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:59.215629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45646 (local address 127.27.77.193:46279)
0806 00:56:59.216456 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:59.216465 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:59.216505 (+ 40us) server_negotiation.cc:407] Connection header received
0806 00:56:59.216590 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:59.216599 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:59.216815 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:59.217379 (+ 564us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:59.217535 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.218378 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.220054 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.220905 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.221266 (+ 361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:59.221410 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:59.221470 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 00:56:59.221582 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":102,"threads_started":1}
I20260806 00:56:59.222010 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:59.216248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:36079 (local address 127.0.0.1:47032)
0806 00:56:59.216525 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:59.216539 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:59.216624 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:59.217410 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:59.217413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:59.217429 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:59.217510 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:59.217518 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.218903 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.218906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:59.220478 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:59.220486 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.221591 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.221594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:59.221676 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:59.221699 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:56:59.221782 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 00:56:59.221861 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":415,"thread_start_us":105,"threads_started":1}
I20260806 00:56:59.222421 16766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:59.215597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:37553 (local address 127.0.0.1:42902)
0806 00:56:59.216007 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:59.216023 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:59.216166 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:59.217686 (+ 1520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:59.217689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:59.217707 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:59.217812 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:59.217818 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.219238 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.219244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:59.220335 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:59.220346 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.221210 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.221214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:59.222148 (+ 934us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:59.222181 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 00:56:59.222258 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 00:56:59.222325 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":4,"spinlock_wait_cycles":8064,"thread_start_us":131,"threads_started":1}
I20260806 00:56:59.222548 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:59.216335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42902 (local address 127.27.77.194:37553)
0806 00:56:59.216844 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:59.216857 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:59.216880 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:56:59.216967 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:59.216972 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:59.217152 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:59.217308 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:59.218076 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.219112 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.220492 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.221077 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.221209 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:59.221379 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:59.222309 (+ 930us) server_negotiation.cc:299] Negotiation successful
0806 00:56:59.222410 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":138,"threads_started":1}
I20260806 00:56:59.223371 16770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:59.216675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47032 (local address 127.27.77.195:36079)
0806 00:56:59.216998 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:59.217009 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:59.217029 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:56:59.217095 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:59.217099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:59.217256 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:59.217432 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:59.217614 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.218759 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.220624 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.221457 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.221578 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:59.221702 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:59.223103 (+ 1401us) server_negotiation.cc:299] Negotiation successful
0806 00:56:59.223214 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":105,"threads_started":1}
I20260806 00:56:59.224985 16300 tablet_service.cc:1467] Processing CreateTablet for tablet 99526d78374a487584a4361ad27703e4 (DEFAULT_TABLE table=test-table [id=646347d1dfbf4897a6a7f84ba5314183]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:56:59.225672 16657 tablet_service.cc:1467] Processing CreateTablet for tablet 99526d78374a487584a4361ad27703e4 (DEFAULT_TABLE table=test-table [id=646347d1dfbf4897a6a7f84ba5314183]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:56:59.225991 16300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99526d78374a487584a4361ad27703e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:59.226502 16657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99526d78374a487584a4361ad27703e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:59.229449 16476 tablet_service.cc:1467] Processing CreateTablet for tablet 99526d78374a487584a4361ad27703e4 (DEFAULT_TABLE table=test-table [id=646347d1dfbf4897a6a7f84ba5314183]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:56:59.230649 16476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99526d78374a487584a4361ad27703e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:56:59.233134 16771 tablet_bootstrap.cc:492] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: Bootstrap starting.
I20260806 00:56:59.235450 16771 tablet_bootstrap.cc:654] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:59.236220 16771 log.cc:824] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:59.237306 16777 tablet_bootstrap.cc:492] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c: Bootstrap starting.
I20260806 00:56:59.238284 16772 tablet_bootstrap.cc:492] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Bootstrap starting.
I20260806 00:56:59.238700 16771 tablet_bootstrap.cc:492] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: No bootstrap required, opened a new log
I20260806 00:56:59.238912 16771 ts_tablet_manager.cc:1397] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:56:59.240054 16777 tablet_bootstrap.cc:654] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:59.240476 16772 tablet_bootstrap.cc:654] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Neither blocks nor log segments found. Creating new log.
I20260806 00:56:59.241046 16777 log.cc:824] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:59.241725 16772 log.cc:824] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:56:59.243191 16777 tablet_bootstrap.cc:492] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c: No bootstrap required, opened a new log
I20260806 00:56:59.243115 16771 raft_consensus.cc:348] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } } peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } }
I20260806 00:56:59.243393 16777 ts_tablet_manager.cc:1397] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:56:59.243417 16771 raft_consensus.cc:374] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:59.243552 16771 raft_consensus.cc:729] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a185f155050e4e458a419bf1e80b84e6, State: Initialized, Role: FOLLOWER
I20260806 00:56:59.243794 16772 tablet_bootstrap.cc:492] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c: No bootstrap required, opened a new log
I20260806 00:56:59.243981 16772 ts_tablet_manager.cc:1397] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:56:59.244158 16771 consensus_queue.cc:260] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } } peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } }
I20260806 00:56:59.245154 16783 raft_consensus.cc:482] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:56:59.245343 16783 raft_consensus.cc:504] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } } peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } }
I20260806 00:56:59.245743 16771 ts_tablet_manager.cc:1428] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 00:56:59.246145 16384 heartbeater.cc:503] Master 127.27.77.254:38001 was elected leader, sending a full tablet report...
I20260806 00:56:59.246631 16783 leader_election.cc:288] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553), ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079)
I20260806 00:56:59.248028 16772 raft_consensus.cc:348] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } } peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } }
I20260806 00:56:59.248310 16772 raft_consensus.cc:374] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:59.248384 16772 raft_consensus.cc:729] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef870f4eb07a49a2bbad1b1fd1f3d08c, State: Initialized, Role: FOLLOWER
I20260806 00:56:59.248847 16772 consensus_queue.cc:260] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } } peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } }
I20260806 00:56:59.249018 16777 raft_consensus.cc:348] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } } peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } }
I20260806 00:56:59.249287 16777 raft_consensus.cc:374] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:56:59.249348 16777 raft_consensus.cc:729] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d3adbf2316b4d47a1465ee793d9e85c, State: Initialized, Role: FOLLOWER
I20260806 00:56:59.249684 16746 heartbeater.cc:503] Master 127.27.77.254:38001 was elected leader, sending a full tablet report...
I20260806 00:56:59.249892 16772 ts_tablet_manager.cc:1428] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:56:59.250270 16777 consensus_queue.cc:260] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } } peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } }
I20260806 00:56:59.251415 16777 ts_tablet_manager.cc:1428] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 00:56:59.251761 16565 heartbeater.cc:503] Master 127.27.77.254:38001 was elected leader, sending a full tablet report...
I20260806 00:56:59.254714 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:59.246899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:43453 (local address 127.27.77.194:37553)
0806 00:56:59.247035 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:59.247040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:59.250337 (+ 3297us) server_negotiation.cc:407] Connection header received
0806 00:56:59.250569 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:59.250574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:59.250627 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:59.250679 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:59.251134 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.251890 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.253464 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.254006 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.254080 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:59.254119 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:59.254511 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 00:56:59.254592 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:56:59.254750 16784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:59.246849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:37553 (local address 127.27.77.193:43453)
0806 00:56:59.250237 (+ 3388us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:59.250253 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:59.250416 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:59.250824 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:59.250827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:59.250845 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:59.250990 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:59.250998 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.252002 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.252005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:59.253243 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:59.253255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.254235 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.254238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:59.254348 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:59.254374 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:56:59.254488 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 00:56:59.254575 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3319,"thread_start_us":108,"threads_started":1}
I20260806 00:56:59.255324 16507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99526d78374a487584a4361ad27703e4" candidate_uuid: "a185f155050e4e458a419bf1e80b84e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" is_pre_election: true
I20260806 00:56:59.255657 16507 raft_consensus.cc:2454] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a185f155050e4e458a419bf1e80b84e6 in term 0.
I20260806 00:56:59.256152 16238 leader_election.cc:302] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d3adbf2316b4d47a1465ee793d9e85c, a185f155050e4e458a419bf1e80b84e6; no voters:
I20260806 00:56:59.256434 16783 raft_consensus.cc:2781] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:56:59.256547 16783 raft_consensus.cc:482] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:56:59.256603 16783 raft_consensus.cc:3037] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:59.257091 16770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:59.247366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:44137 (local address 127.27.77.195:36079)
0806 00:56:59.247472 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:59.247476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:59.251291 (+ 3815us) server_negotiation.cc:407] Connection header received
0806 00:56:59.251473 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:59.251477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:59.251554 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:59.251603 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:56:59.253445 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.254103 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.255818 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.256410 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.256517 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:59.256578 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:59.256934 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 00:56:59.257000 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 00:56:59.257148 16786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:59.247271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:36079 (local address 127.27.77.193:44137)
0806 00:56:59.251210 (+ 3939us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:59.251224 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:56:59.251353 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:56:59.251746 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:56:59.251748 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:56:59.251765 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:56:59.251883 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:59.251890 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.254244 (+ 2354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.254248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:59.255670 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:56:59.255684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.256686 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.256689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:56:59.256792 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:59.256829 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 00:56:59.256910 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 00:56:59.256991 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3875,"thread_start_us":91,"threads_started":1}
I20260806 00:56:59.257884 16686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99526d78374a487584a4361ad27703e4" candidate_uuid: "a185f155050e4e458a419bf1e80b84e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" is_pre_election: true
I20260806 00:56:59.258132 16783 raft_consensus.cc:504] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } } peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } }
I20260806 00:56:59.258189 16686 raft_consensus.cc:2454] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a185f155050e4e458a419bf1e80b84e6 in term 0.
I20260806 00:56:59.258553 16783 leader_election.cc:288] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 1 election: Requested vote from peers 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553), ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079)
I20260806 00:56:59.258898 16507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99526d78374a487584a4361ad27703e4" candidate_uuid: "a185f155050e4e458a419bf1e80b84e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d3adbf2316b4d47a1465ee793d9e85c"
I20260806 00:56:59.259029 16507 raft_consensus.cc:3037] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:59.259284 16686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99526d78374a487584a4361ad27703e4" candidate_uuid: "a185f155050e4e458a419bf1e80b84e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c"
I20260806 00:56:59.259379 16686 raft_consensus.cc:3037] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:56:59.260236 16507 raft_consensus.cc:2454] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a185f155050e4e458a419bf1e80b84e6 in term 1.
I20260806 00:56:59.260588 16686 raft_consensus.cc:2454] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a185f155050e4e458a419bf1e80b84e6 in term 1.
I20260806 00:56:59.260644 16238 leader_election.cc:302] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d3adbf2316b4d47a1465ee793d9e85c, a185f155050e4e458a419bf1e80b84e6; no voters:
I20260806 00:56:59.260830 16783 raft_consensus.cc:2781] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:56:59.261042 16783 raft_consensus.cc:686] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [term 1 LEADER]: Becoming Leader. State: Replica: a185f155050e4e458a419bf1e80b84e6, State: Running, Role: LEADER
I20260806 00:56:59.261376 16783 consensus_queue.cc:237] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } } peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } }
I20260806 00:56:59.264919 16126 catalog_manager.cc:5495] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a185f155050e4e458a419bf1e80b84e6 (127.27.77.193). New cstate: current_term: 1 leader_uuid: "a185f155050e4e458a419bf1e80b84e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } health_report { overall_health: UNKNOWN } } }
I20260806 00:56:59.286844 16767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:59.282988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45662 (local address 127.27.77.193:46279)
0806 00:56:59.283134 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:56:59.283138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:56:59.283151 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:56:59.283229 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:56:59.283231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:56:59.283285 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:56:59.283371 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:56:59.283772 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.284304 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.285242 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:56:59.285470 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:56:59.285563 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:56:59.285983 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:56:59.286409 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:56:59.286512 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:56:59.286666 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 00:56:59.286727 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 00:56:59.293747 16686 raft_consensus.cc:1261] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 1 FOLLOWER]: Refusing update from remote peer a185f155050e4e458a419bf1e80b84e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:56:59.293747 16507 raft_consensus.cc:1261] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c [term 1 FOLLOWER]: Refusing update from remote peer a185f155050e4e458a419bf1e80b84e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:56:59.294560 16783 consensus_queue.cc:1035] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:56:59.294777 16780 consensus_queue.cc:1035] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:56:59.304987 16795 mvcc.cc:205] Tried to move back new op lower bound from 7315365147818778624 to 7315365147697938432. Current Snapshot: MvccSnapshot[applied={T|T < 7315365147818778624}]
I20260806 00:56:59.313378 16126 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:56:59.313570 16126 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:56:59.316363 16126 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Sending DeleteTablet for 3 replicas of tablet 99526d78374a487584a4361ad27703e4
I20260806 00:56:59.320119 16476 tablet_service.cc:1514] Processing DeleteTablet for tablet 99526d78374a487584a4361ad27703e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:56:59 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:42902
I20260806 00:56:59.320684 16657 tablet_service.cc:1514] Processing DeleteTablet for tablet 99526d78374a487584a4361ad27703e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:56:59 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:47032
I20260806 00:56:59.321211 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 16215
I20260806 00:56:59.325237 16805 tablet_replica.cc:330] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c: stopping tablet replica
I20260806 00:56:59.325809 16805 raft_consensus.cc:2229] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:56:59.326335 16805 raft_consensus.cc:2258] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:56:59.327878 16807 tablet_replica.cc:330] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c: stopping tablet replica
I20260806 00:56:59.328392 16807 raft_consensus.cc:2229] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:56:59.328858 16807 raft_consensus.cc:2258] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:56:59.330138 16805 ts_tablet_manager.cc:1905] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:56:59.331800 16807 ts_tablet_manager.cc:1905] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:56:59.333801 16805 ts_tablet_manager.cc:1918] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:56:59.333925 16805 log.cc:1196] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/99526d78374a487584a4361ad27703e4
I20260806 00:56:59.334365 16805 ts_tablet_manager.cc:1939] T 99526d78374a487584a4361ad27703e4 P 0d3adbf2316b4d47a1465ee793d9e85c: Deleting consensus metadata
I20260806 00:56:59.335561 16807 ts_tablet_manager.cc:1918] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:56:59.335707 16807 log.cc:1196] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/99526d78374a487584a4361ad27703e4
I20260806 00:56:59.336048 16110 catalog_manager.cc:4841] TS 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553): tablet 99526d78374a487584a4361ad27703e4 (table test-table [id=646347d1dfbf4897a6a7f84ba5314183]) successfully deleted
I20260806 00:56:59.336143 16807 ts_tablet_manager.cc:1939] T 99526d78374a487584a4361ad27703e4 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Deleting consensus metadata
I20260806 00:56:59.337957 16110 catalog_manager.cc:4841] TS ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079): tablet 99526d78374a487584a4361ad27703e4 (table test-table [id=646347d1dfbf4897a6a7f84ba5314183]) successfully deleted
W20260806 00:56:59.348172 16108 connection.cc:535] client connection to 127.27.77.193:46279 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 00:56:59.348192 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 16390
W20260806 00:56:59.348392 16108 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 00:56:59.356001 16766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:56:59.355444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:46279 (local address 127.0.0.1:45664)
0806 00:56:59.355726 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 00:56:59.355855 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:46279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260806 00:56:59.356302 16108 catalog_manager.cc:4584] TS a185f155050e4e458a419bf1e80b84e6 (127.27.77.193:46279): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 99526d78374a487584a4361ad27703e4: Network error: Client connection negotiation failed: client connection to 127.27.77.193:46279: connect: Connection refused (error 111)
I20260806 00:56:59.366245 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 16573
I20260806 00:56:59.387370 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 16084
I20260806 00:57:19.397862 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38001
--webserver_interface=127.27.77.254
--webserver_port=46105
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:38001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:57:19.514412 16809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:19.514705 16809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:19.514750 16809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:19.518875 16809 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 00:57:19.518932 16809 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 00:57:19.518949 16809 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 00:57:19.518962 16809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:57:19.518980 16809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:19.519006 16809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:19.519069 16809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:57:19.519090 16809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:57:19.524766 16809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:38001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=46105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:57:19.526011 16809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:57:19.527086 16809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:57:19.542157 16816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:19.542040 16817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:19.542040 16819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:19.542520 16809 server_base.cc:1034] running on GCE node
I20260806 00:57:19.543267 16809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:57:19.544713 16809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:57:19.545923 16809 hybrid_clock.cc:648] HybridClock initialized: now 1785977839545882 us; error 58 us; skew 500 ppm
Aug 06 00:57:19 dist-test-slave-sb2z krb5kdc[16063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977839, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:57:19.549346 16809 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:57:19.550486 16809 webserver.cc:460] Webserver started at http://127.27.77.254:46105/ using document root <none> and password file <none>
I20260806 00:57:19.551105 16809 fs_manager.cc:359] Metadata directory not provided
I20260806 00:57:19.551188 16809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:57:19.556277 16809 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:57:19.558712 16826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:19.560177 16809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:57:19.560331 16809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "6a0f517681684136a90430c795ded7cc"
format_stamp: "Formatted at 2026-08-06 00:56:58 on dist-test-slave-sb2z"
server_key: "2e466672b1492ea07ca2ef943c6068d5"
server_key_iv: "bf57644131e95e3cf9b73799196fa4d8"
server_key_version: "encryptionkey@0"
I20260806 00:57:19.560984 16809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:57:19.575333 16809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:57:19.578768 16809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:57:19.579010 16809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:57:19.587299 16809 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:38001
I20260806 00:57:19.587778 16902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:38001 every 8 connection(s)
I20260806 00:57:19.588618 16809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:57:19.593940 16904 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:57:19.596369 16904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Bootstrap starting.
I20260806 00:57:19.596626 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 16809
I20260806 00:57:19.597234 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:46279
--local_ip_for_outbound_sockets=127.27.77.193
--tserver_master_addrs=127.27.77.254:38001
--webserver_port=43965
--webserver_interface=127.27.77.193
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:57:19.617761 16904 log.cc:824] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Log is configured to *not* fsync() on all Append() calls
I20260806 00:57:19.626890 16904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:57:19.627321 16904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Bootstrap complete.
I20260806 00:57:19.631584 16904 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } }
I20260806 00:57:19.632010 16904 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a0f517681684136a90430c795ded7cc, State: Initialized, Role: FOLLOWER
I20260806 00:57:19.632565 16904 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } }
I20260806 00:57:19.632735 16904 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:57:19.632845 16904 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:57:19.632992 16904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:57:19.634974 16904 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } }
I20260806 00:57:19.635340 16904 leader_election.cc:302] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a0f517681684136a90430c795ded7cc; no voters:
I20260806 00:57:19.635736 16904 leader_election.cc:288] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 00:57:19.635834 16911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 2 FOLLOWER]: Leader election won for term 2
I20260806 00:57:19.636041 16911 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 2 LEADER]: Becoming Leader. State: Replica: 6a0f517681684136a90430c795ded7cc, State: Running, Role: LEADER
I20260806 00:57:19.636404 16911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } }
I20260806 00:57:19.637095 16904 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:57:19.637858 16912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6a0f517681684136a90430c795ded7cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } } }
I20260806 00:57:19.638006 16912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [sys.catalog]: This master's current role is: LEADER
I20260806 00:57:19.638450 16914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a0f517681684136a90430c795ded7cc. Latest consensus state: current_term: 2 leader_uuid: "6a0f517681684136a90430c795ded7cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } } }
I20260806 00:57:19.638598 16914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [sys.catalog]: This master's current role is: LEADER
I20260806 00:57:19.639197 16917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:57:19.644116 16917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:57:19.644887 16917 catalog_manager.cc:1259] Loaded cluster ID: c159e7cb46da44bf9f56636be35754f6
I20260806 00:57:19.644958 16917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:57:19.647562 16917 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:57:19.648365 16917 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Loaded TSK: 0
I20260806 00:57:19.667321 16917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Generated new TSK 1
I20260806 00:57:19.667667 16917 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:57:19.717864 16909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:19.718180 16909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:19.718267 16909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:19.722361 16909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:19.722468 16909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:19.722560 16909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:57:19.728256 16909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:46279
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=43965
--tserver_master_addrs=127.27.77.254:38001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:57:19.729374 16909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:57:19.730634 16909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:57:19.741292 16943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:19.741240 16941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:19.741689 16948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:19.741686 16909 server_base.cc:1034] running on GCE node
I20260806 00:57:19.742143 16909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:57:19.742864 16909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:57:19.744053 16909 hybrid_clock.cc:648] HybridClock initialized: now 1785977839744020 us; error 53 us; skew 500 ppm
Aug 06 00:57:19 dist-test-slave-sb2z krb5kdc[16063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977839, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:57:19.747102 16909 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:57:19.748206 16909 webserver.cc:460] Webserver started at http://127.27.77.193:43965/ using document root <none> and password file <none>
I20260806 00:57:19.748808 16909 fs_manager.cc:359] Metadata directory not provided
I20260806 00:57:19.748888 16909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:57:19.752995 16909 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:57:19.755388 16958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:19.756815 16909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:57:19.756968 16909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "a185f155050e4e458a419bf1e80b84e6"
format_stamp: "Formatted at 2026-08-06 00:56:58 on dist-test-slave-sb2z"
server_key: "1da3955ee1ba2127471670fb38cdfa3d"
server_key_iv: "a86a44c4dae836089752fc4c615c1e3d"
server_key_version: "encryptionkey@0"
I20260806 00:57:19.757573 16909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:57:19.771065 16909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:57:19.774544 16909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:57:19.774786 16909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:57:19.775606 16909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:57:19.776794 16966 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 00:57:19.782536 16909 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 00:57:19.782629 16909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260806 00:57:19.782714 16909 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 00:57:19.784543 16909 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 00:57:19.784730 16909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 00:57:19.784929 16966 tablet_bootstrap.cc:492] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: Bootstrap starting.
I20260806 00:57:19.796368 16909 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:46279
I20260806 00:57:19.796641 17112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:46279 every 8 connection(s)
I20260806 00:57:19.797509 16909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:57:19.805830 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 16909
I20260806 00:57:19.806393 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:37553
--local_ip_for_outbound_sockets=127.27.77.194
--tserver_master_addrs=127.27.77.254:38001
--webserver_port=38537
--webserver_interface=127.27.77.194
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:57:19.807971 16966 log.cc:824] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:57:19 dist-test-slave-sb2z krb5kdc[16063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977839, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:57:19.816221 17117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:19.799736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:56747 (local address 127.27.77.254:38001)
0806 00:57:19.800809 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:19.800829 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:19.801258 (+ 429us) server_negotiation.cc:407] Connection header received
0806 00:57:19.802416 (+ 1158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:19.802426 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:19.802737 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:19.803088 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:19.804336 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:19.805300 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:19.807067 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:19.807814 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:19.808042 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:19.810805 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:19.810812 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:19.810830 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:57:19.810864 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:19.812847 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:19.813448 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:19.813452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:19.813456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:19.813578 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:19.813945 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:19.813948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:19.813950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:19.814308 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:19.814456 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:19.814990 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 00:57:19.815256 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":930,"thread_start_us":111,"threads_started":1}
I20260806 00:57:19.816236 17119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:19.799776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:56747)
0806 00:57:19.801080 (+ 1304us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:19.801122 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:19.801991 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:19.803133 (+ 1142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:19.803141 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:19.803670 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:57:19.804149 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:19.804164 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:19.805487 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:19.805490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:19.806901 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:19.806917 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:19.807986 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:19.807993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:19.808232 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:19.809031 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:57:19.809057 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:57:19.810574 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:57:19.813000 (+ 2426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:19.813006 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:19.813020 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:19.813300 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:19.813690 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:19.813695 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:19.813697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:19.813840 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:19.814438 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:57:19.814444 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:57:19.814686 (+ 242us) client_negotiation.cc:769] Sending connection context
0806 00:57:19.814876 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 00:57:19.815243 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1110,"thread_start_us":140,"threads_started":1}
I20260806 00:57:19.818342 16966 tablet_bootstrap.cc:492] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:57:19.818878 16966 tablet_bootstrap.cc:492] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: Bootstrap complete.
I20260806 00:57:19.819027 17115 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38001
I20260806 00:57:19.819274 17115 heartbeater.cc:464] Registering TS with master...
I20260806 00:57:19.819844 17115 heartbeater.cc:511] Master 127.27.77.254:38001 requested a full tablet report, sending...
I20260806 00:57:19.820710 16966 ts_tablet_manager.cc:1397] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: Time spent bootstrapping tablet: real 0.036s user 0.026s sys 0.008s
I20260806 00:57:19.822286 16847 ts_manager.cc:194] Registered new tserver with Master: a185f155050e4e458a419bf1e80b84e6 (127.27.77.193:46279)
I20260806 00:57:19.824128 16966 raft_consensus.cc:348] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } } peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } }
I20260806 00:57:19.824682 16966 raft_consensus.cc:729] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a185f155050e4e458a419bf1e80b84e6, State: Initialized, Role: FOLLOWER
I20260806 00:57:19.825219 16966 consensus_queue.cc:260] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } } peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } }
I20260806 00:57:19.826323 16966 ts_tablet_manager.cc:1428] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:57:19.826722 16847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:56747
I20260806 00:57:19.829378 17115 heartbeater.cc:503] Master 127.27.77.254:38001 was elected leader, sending a full tablet report...
Aug 06 00:57:19 dist-test-slave-sb2z krb5kdc[16063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977839, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for kudu/127.27.77.193@KRBTEST.COM
I20260806 00:57:19.838054 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:19.825360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:46279 (local address 127.0.0.1:41740)
0806 00:57:19.827044 (+ 1684us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:19.827099 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:19.827239 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:19.828033 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:19.828039 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:19.828490 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:57:19.828635 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:19.828644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:19.830115 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:19.830118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:19.831027 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:19.831040 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:19.831764 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:19.831767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:19.831885 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:19.832528 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:57:19.832550 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:57:19.834010 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:57:19.836068 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:19.836080 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:19.836086 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:19.836327 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:19.836661 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:19.836664 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:19.836666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:19.836817 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:19.837348 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:57:19.837355 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:57:19.837485 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 00:57:19.837641 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 00:57:19.837855 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1491,"thread_start_us":148,"threads_started":1}
I20260806 00:57:19.838157 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:19.825404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41740 (local address 127.27.77.193:46279)
0806 00:57:19.827560 (+ 2156us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:19.827573 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:19.827595 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:57:19.827660 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:19.827665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:19.827862 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:19.827993 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:19.828778 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:19.830012 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:19.831179 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:19.831635 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:19.831760 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:19.834162 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:19.834172 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:19.834186 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:57:19.834217 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:19.835947 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:19.836461 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:19.836466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:19.836468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:19.836550 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:19.836945 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:19.836948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:19.836950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:19.837217 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:19.837365 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:19.837730 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 00:57:19.837980 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2011,"thread_start_us":108,"threads_started":1}
I20260806 00:57:19.839212 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet 99526d78374a487584a4361ad27703e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:56:59 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:41740
I20260806 00:57:19.840303 17127 tablet_replica.cc:330] stopping tablet replica
I20260806 00:57:19.840652 17127 raft_consensus.cc:2229] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:57:19.840945 17127 raft_consensus.cc:2258] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:57:19.842104 17127 ts_tablet_manager.cc:1905] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:57:19.846729 17127 ts_tablet_manager.cc:1918] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:57:19.846839 17127 log.cc:1196] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal/wals/99526d78374a487584a4361ad27703e4
I20260806 00:57:19.847127 17127 ts_tablet_manager.cc:1939] T 99526d78374a487584a4361ad27703e4 P a185f155050e4e458a419bf1e80b84e6: Deleting consensus metadata
I20260806 00:57:19.849493 16827 catalog_manager.cc:4841] TS a185f155050e4e458a419bf1e80b84e6 (127.27.77.193:46279): tablet 99526d78374a487584a4361ad27703e4 (table test-table [id=646347d1dfbf4897a6a7f84ba5314183]) successfully deleted
W20260806 00:57:19.934933 17120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:19.935218 17120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:19.935264 17120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:19.939687 17120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:19.939775 17120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:19.939891 17120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:57:19.945952 17120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:37553
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=38537
--tserver_master_addrs=127.27.77.254:38001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:57:19.947068 17120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:57:19.948171 17120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:57:19.958279 17139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:19.958274 17137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:19.958271 17135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:19.958724 17120 server_base.cc:1034] running on GCE node
I20260806 00:57:19.959165 17120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:57:19.959889 17120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:57:19.961081 17120 hybrid_clock.cc:648] HybridClock initialized: now 1785977839961060 us; error 36 us; skew 500 ppm
Aug 06 00:57:19 dist-test-slave-sb2z krb5kdc[16063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977839, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:57:19.964264 17120 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:57:19.965248 17120 webserver.cc:460] Webserver started at http://127.27.77.194:38537/ using document root <none> and password file <none>
I20260806 00:57:19.965907 17120 fs_manager.cc:359] Metadata directory not provided
I20260806 00:57:19.966064 17120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:57:19.970211 17120 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:57:19.972436 17147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:19.973886 17120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:57:19.974095 17120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "0d3adbf2316b4d47a1465ee793d9e85c"
format_stamp: "Formatted at 2026-08-06 00:56:58 on dist-test-slave-sb2z"
server_key: "aa47892fa16621b2df76b0e260d86c76"
server_key_iv: "ab16397e0741c817d618f3918c502ff0"
server_key_version: "encryptionkey@0"
I20260806 00:57:19.974673 17120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:57:19.994534 17120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:57:19.998060 17120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:57:19.998284 17120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:57:19.998936 17120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:57:19.999842 17120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:57:19.999915 17120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:19.999991 17120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:57:20.000036 17120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:20.010583 17120 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:37553
I20260806 00:57:20.010607 17296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:37553 every 8 connection(s)
I20260806 00:57:20.011718 17120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:57:20.017061 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 17120
I20260806 00:57:20.017542 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:36079
--local_ip_for_outbound_sockets=127.27.77.195
--tserver_master_addrs=127.27.77.254:38001
--webserver_port=43301
--webserver_interface=127.27.77.195
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:57:20 dist-test-slave-sb2z krb5kdc[16063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977839, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:57:20.027212 17117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.013604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:47485 (local address 127.27.77.254:38001)
0806 00:57:20.013796 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:20.013801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:20.014839 (+ 1038us) server_negotiation.cc:407] Connection header received
0806 00:57:20.015752 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:20.015756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:20.015811 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:20.015890 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:20.017250 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.017902 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.019728 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.020219 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.020345 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.023220 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:20.023227 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:20.023233 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:57:20.023267 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:20.025047 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:20.025625 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:20.025628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:20.025629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:20.025707 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:20.026059 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:20.026063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:20.026065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:20.026262 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:20.026376 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:20.026872 (+ 496us) server_negotiation.cc:299] Negotiation successful
0806 00:57:20.027031 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 00:57:20.028095 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.014017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:47485)
0806 00:57:20.014678 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:20.014720 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:20.015492 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:20.016054 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:20.016060 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:20.016447 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:57:20.017049 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:20.017073 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.018036 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.018043 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:20.019552 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:20.019569 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.020397 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.020399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:20.020542 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.021216 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:57:20.021245 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:57:20.023025 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:57:20.025193 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:20.025200 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:20.025216 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:20.025495 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:20.025806 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:20.025809 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:20.025811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:20.025958 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:20.026376 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:57:20.026384 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:57:20.026634 (+ 250us) client_negotiation.cc:769] Sending connection context
0806 00:57:20.026830 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 00:57:20.027165 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":121,"threads_started":1}
I20260806 00:57:20.029331 17297 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38001
I20260806 00:57:20.029610 17297 heartbeater.cc:464] Registering TS with master...
I20260806 00:57:20.030267 17297 heartbeater.cc:511] Master 127.27.77.254:38001 requested a full tablet report, sending...
I20260806 00:57:20.031491 16847 ts_manager.cc:194] Registered new tserver with Master: 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553)
I20260806 00:57:20.032272 16847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:47485
W20260806 00:57:20.136544 17300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:20.136803 17300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:20.136847 17300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:20.141338 17300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:20.141400 17300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:20.141460 17300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:57:20.146718 17300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:36079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=43301
--tserver_master_addrs=127.27.77.254:38001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:57:20.147804 17300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:57:20.148814 17300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:57:20.158797 17310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:20.158859 17308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:20.158993 17300 server_base.cc:1034] running on GCE node
W20260806 00:57:20.158816 17307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:20.159593 17300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:57:20.160272 17300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:57:20.161474 17300 hybrid_clock.cc:648] HybridClock initialized: now 1785977840161459 us; error 25 us; skew 500 ppm
Aug 06 00:57:20 dist-test-slave-sb2z krb5kdc[16063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977840, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:57:20.164512 17300 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:57:20.165648 17300 webserver.cc:460] Webserver started at http://127.27.77.195:43301/ using document root <none> and password file <none>
I20260806 00:57:20.166278 17300 fs_manager.cc:359] Metadata directory not provided
I20260806 00:57:20.166366 17300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:57:20.170673 17300 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:57:20.173017 17320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:20.174307 17300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:57:20.174464 17300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c"
format_stamp: "Formatted at 2026-08-06 00:56:59 on dist-test-slave-sb2z"
server_key: "49c3870aa3e743f9166b3c1937e68a1d"
server_key_iv: "b99df649d2eaecc14ddbf37c26dcfe83"
server_key_version: "encryptionkey@0"
I20260806 00:57:20.175051 17300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:57:20.195926 17300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:57:20.199416 17300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:57:20.199618 17300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:57:20.200217 17300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:57:20.201047 17300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:57:20.201095 17300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:20.201135 17300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:57:20.201150 17300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:20.211118 17300 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:36079
I20260806 00:57:20.211145 17469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:36079 every 8 connection(s)
I20260806 00:57:20.212183 17300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:57:20.216620 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 17300
Aug 06 00:57:20 dist-test-slave-sb2z krb5kdc[16063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977840, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:57:20.225631 17474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.217398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48960 (local address 127.27.77.254:38001)
0806 00:57:20.217957 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:20.217961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:20.217972 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:57:20.218003 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:20.218006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:20.218039 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:20.218143 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:20.218890 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.219399 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.220269 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.220497 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.220619 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.222080 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:20.222088 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:20.222090 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:57:20.222113 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:20.223848 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:20.224345 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:20.224350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:20.224352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:20.224447 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:20.224740 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:20.224744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:20.224745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:20.224932 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:20.225157 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:20.225294 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 00:57:20.225417 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":117,"threads_started":1}
I20260806 00:57:20.226912 17117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.214217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:37035 (local address 127.27.77.254:38001)
0806 00:57:20.214387 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:20.214391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:20.215338 (+ 947us) server_negotiation.cc:407] Connection header received
0806 00:57:20.216799 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:20.216803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:20.216869 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:20.216983 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:20.218350 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.218876 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.220109 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.220331 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.220457 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.223279 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:20.223283 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:20.223285 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:57:20.223331 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:20.224749 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:20.225286 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:20.225291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:20.225293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:20.225373 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:20.225738 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:20.225743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:20.225744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:20.225937 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:20.226035 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:20.226622 (+ 587us) server_negotiation.cc:299] Negotiation successful
0806 00:57:20.226746 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 00:57:20.227949 17472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.214561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:37035)
0806 00:57:20.215163 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:20.215208 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:20.216556 (+ 1348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:20.217180 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:20.217187 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:20.217700 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:57:20.218198 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:20.218212 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.219016 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.219019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:20.219980 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:20.219991 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.220490 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.220494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:20.220666 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.221419 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:57:20.221437 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:57:20.223097 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:57:20.224878 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:20.224884 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:20.224899 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:20.225175 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:20.225483 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:20.225486 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:20.225488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:20.225637 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:20.226043 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:57:20.226048 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:57:20.226365 (+ 317us) client_negotiation.cc:769] Sending connection context
0806 00:57:20.226540 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 00:57:20.226879 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":125,"threads_started":1}
I20260806 00:57:20.229092 17470 heartbeater.cc:347] Connected to a master server at 127.27.77.254:38001
I20260806 00:57:20.229342 17470 heartbeater.cc:464] Registering TS with master...
I20260806 00:57:20.229843 17470 heartbeater.cc:511] Master 127.27.77.254:38001 requested a full tablet report, sending...
I20260806 00:57:20.230894 16847 ts_manager.cc:194] Registered new tserver with Master: ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079)
I20260806 00:57:20.231477 16847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:37035
I20260806 00:57:20.239825 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:57:20.244179 17117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.240530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48962 (local address 127.27.77.254:38001)
0806 00:57:20.240675 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:20.240679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:20.240852 (+ 173us) server_negotiation.cc:407] Connection header received
0806 00:57:20.241075 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:20.241078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:20.241186 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:20.241294 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:57:20.241523 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.242090 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.243001 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.243212 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.243295 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.243662 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:57:20.243803 (+ 141us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 00:57:20.244077 (+ 274us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48962: token expired
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 00:57:20.244287 17117 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48962: token expired
W20260806 00:57:20.244670 16758 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.27.77.254:38001)
I20260806 00:57:20.252337 17117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.245094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48978 (local address 127.27.77.254:38001)
0806 00:57:20.245242 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:20.245246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:20.245333 (+ 87us) server_negotiation.cc:407] Connection header received
0806 00:57:20.245452 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:20.245455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:20.245502 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:20.245592 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:20.246241 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.246752 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.247595 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.247889 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.247994 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.249334 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:20.249338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:20.249341 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:57:20.249366 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:20.250634 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:20.251070 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:20.251073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:20.251075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:20.251160 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:20.251437 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:20.251442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:20.251443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:20.251679 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:20.251819 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:20.252052 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 00:57:20.252165 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:57:20.254685 16847 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:57:20.258107 16847 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:57:20.266824 17021 tablet_service.cc:1467] Processing CreateTablet for tablet 6f39e989849a44d58ff4d78bd4c57e36 (DEFAULT_TABLE table=test-table [id=b1837c8680be42a2b794e6cf6ae200c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:57:20.267647 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f39e989849a44d58ff4d78bd4c57e36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:57:20.270833 16966 tablet_bootstrap.cc:492] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6: Bootstrap starting.
I20260806 00:57:20.271202 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.265439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:37553 (local address 127.0.0.1:58536)
0806 00:57:20.265581 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:20.265598 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:20.265686 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:20.267083 (+ 1397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:20.267087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:20.267109 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:57:20.267209 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:20.267216 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.268881 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.268885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:20.269843 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:20.269851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.270739 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.270743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:20.270851 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.270913 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 00:57:20.270986 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 00:57:20.271070 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":1}
I20260806 00:57:20.271432 17483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.265713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:36079 (local address 127.0.0.1:37898)
0806 00:57:20.266113 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:20.266131 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:20.266231 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:20.266826 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:20.266829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:20.266847 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:57:20.266976 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:20.266986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.268291 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.268294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:20.269433 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:20.269444 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.270930 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.270934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:20.271047 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.271074 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 00:57:20.271157 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 00:57:20.271233 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":48,"thread_start_us":133,"threads_started":1}
I20260806 00:57:20.271870 17486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.265953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58536 (local address 127.27.77.194:37553)
0806 00:57:20.266524 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:20.266537 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:20.266562 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:57:20.266643 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:20.266648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:20.266846 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:20.267375 (+ 529us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:57:20.267614 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.268742 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.269966 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.270587 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.271386 (+ 799us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.271550 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:20.271615 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 00:57:20.271733 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":161,"threads_started":1}
I20260806 00:57:20.271876 17484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.265917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37898 (local address 127.27.77.195:36079)
0806 00:57:20.266397 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:20.266407 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:20.266427 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:57:20.266502 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:20.266507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:20.266675 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:20.266846 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:57:20.267111 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.268133 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.269589 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.270560 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.270679 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.270850 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:20.271648 (+ 798us) server_negotiation.cc:299] Negotiation successful
0806 00:57:20.271757 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":145,"threads_started":1}
I20260806 00:57:20.272917 16966 tablet_bootstrap.cc:654] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6: Neither blocks nor log segments found. Creating new log.
I20260806 00:57:20.274619 16966 tablet_bootstrap.cc:492] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6: No bootstrap required, opened a new log
I20260806 00:57:20.274717 16966 ts_tablet_manager.cc:1397] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 00:57:20.275209 16966 raft_consensus.cc:348] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:20.275338 16966 raft_consensus.cc:374] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:57:20.274883 17208 tablet_service.cc:1467] Processing CreateTablet for tablet 6f39e989849a44d58ff4d78bd4c57e36 (DEFAULT_TABLE table=test-table [id=b1837c8680be42a2b794e6cf6ae200c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:57:20.274880 17378 tablet_service.cc:1467] Processing CreateTablet for tablet 6f39e989849a44d58ff4d78bd4c57e36 (DEFAULT_TABLE table=test-table [id=b1837c8680be42a2b794e6cf6ae200c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:57:20.275384 16966 raft_consensus.cc:729] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a185f155050e4e458a419bf1e80b84e6, State: Initialized, Role: FOLLOWER
I20260806 00:57:20.275731 16966 consensus_queue.cc:260] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:20.276041 17208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f39e989849a44d58ff4d78bd4c57e36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:57:20.276072 17378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f39e989849a44d58ff4d78bd4c57e36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:57:20.276726 16966 ts_tablet_manager.cc:1428] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:57:20.283589 17489 tablet_bootstrap.cc:492] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c: Bootstrap starting.
I20260806 00:57:20.283833 17490 tablet_bootstrap.cc:492] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Bootstrap starting.
I20260806 00:57:20.286118 17489 tablet_bootstrap.cc:654] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c: Neither blocks nor log segments found. Creating new log.
I20260806 00:57:20.286413 17490 tablet_bootstrap.cc:654] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Neither blocks nor log segments found. Creating new log.
I20260806 00:57:20.286933 17489 log.cc:824] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:57:20.287192 17490 log.cc:824] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:57:20.288954 17489 tablet_bootstrap.cc:492] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c: No bootstrap required, opened a new log
I20260806 00:57:20.288975 17490 tablet_bootstrap.cc:492] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c: No bootstrap required, opened a new log
I20260806 00:57:20.289186 17489 ts_tablet_manager.cc:1397] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:57:20.289207 17490 ts_tablet_manager.cc:1397] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260806 00:57:20.292641 17490 raft_consensus.cc:348] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:20.292869 17490 raft_consensus.cc:374] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:57:20.292929 17490 raft_consensus.cc:729] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef870f4eb07a49a2bbad1b1fd1f3d08c, State: Initialized, Role: FOLLOWER
I20260806 00:57:20.292747 17489 raft_consensus.cc:348] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:20.293066 17489 raft_consensus.cc:374] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:57:20.293157 17489 raft_consensus.cc:729] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d3adbf2316b4d47a1465ee793d9e85c, State: Initialized, Role: FOLLOWER
I20260806 00:57:20.293413 17490 consensus_queue.cc:260] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:20.293643 17489 consensus_queue.cc:260] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:20.294126 17470 heartbeater.cc:503] Master 127.27.77.254:38001 was elected leader, sending a full tablet report...
I20260806 00:57:20.294344 17297 heartbeater.cc:503] Master 127.27.77.254:38001 was elected leader, sending a full tablet report...
I20260806 00:57:20.294440 17490 ts_tablet_manager.cc:1428] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260806 00:57:20.294678 17489 ts_tablet_manager.cc:1428] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
W20260806 00:57:20.299196 17116 tablet.cc:2369] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:57:20.358780 17494 raft_consensus.cc:482] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:57:20.359002 17494 raft_consensus.cc:504] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:20.360340 17494 leader_election.cc:288] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079), a185f155050e4e458a419bf1e80b84e6 (127.27.77.193:46279)
I20260806 00:57:20.365514 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.360657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:46279 (local address 127.27.77.194:35437)
0806 00:57:20.360791 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:20.360807 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:20.360928 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:20.361519 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:20.361525 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:20.361542 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:57:20.361703 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:20.361712 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.362737 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.362740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:20.364055 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:20.364067 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.365003 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.365009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:20.365139 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.365169 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 00:57:20.365233 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 00:57:20.365308 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 00:57:20.365620 17496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.360793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:35437 (local address 127.27.77.193:46279)
0806 00:57:20.361187 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:20.361192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:20.361208 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:57:20.361269 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:20.361274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:20.361346 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:20.361479 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:57:20.361861 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.362593 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.364203 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.364856 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.364994 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.365257 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:20.365300 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 00:57:20.365432 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":140,"threads_started":1}
I20260806 00:57:20.365685 17484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.360850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:39833 (local address 127.27.77.195:36079)
0806 00:57:20.361034 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:20.361040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:20.361444 (+ 404us) server_negotiation.cc:407] Connection header received
0806 00:57:20.361626 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:20.361630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:20.361686 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:20.361740 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:57:20.362190 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.362869 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.364358 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.364954 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.365073 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.365137 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:20.365439 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 00:57:20.365522 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 00:57:20.366071 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.360665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:36079 (local address 127.27.77.194:39833)
0806 00:57:20.361340 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:20.361352 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:20.361475 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:20.361890 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:20.361892 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:20.361906 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:57:20.362037 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:20.362042 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.362987 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.362991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:20.364240 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:20.364248 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.365196 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.365202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:20.365306 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.365325 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 00:57:20.365841 (+ 516us) client_negotiation.cc:240] Negotiation successful
0806 00:57:20.365928 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":347,"thread_start_us":121,"threads_started":1}
I20260806 00:57:20.366202 17055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f39e989849a44d58ff4d78bd4c57e36" candidate_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a185f155050e4e458a419bf1e80b84e6" is_pre_election: true
I20260806 00:57:20.366581 17055 raft_consensus.cc:2454] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d3adbf2316b4d47a1465ee793d9e85c in term 0.
I20260806 00:57:20.366766 17403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f39e989849a44d58ff4d78bd4c57e36" candidate_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" is_pre_election: true
I20260806 00:57:20.367063 17403 raft_consensus.cc:2454] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d3adbf2316b4d47a1465ee793d9e85c in term 0.
I20260806 00:57:20.367076 17148 leader_election.cc:302] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d3adbf2316b4d47a1465ee793d9e85c, a185f155050e4e458a419bf1e80b84e6; no voters:
I20260806 00:57:20.367354 17494 raft_consensus.cc:2781] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:57:20.367455 17494 raft_consensus.cc:482] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:57:20.367504 17494 raft_consensus.cc:3037] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:57:20.368644 17494 raft_consensus.cc:504] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:20.369015 17494 leader_election.cc:288] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [CANDIDATE]: Term 1 election: Requested vote from peers ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079), a185f155050e4e458a419bf1e80b84e6 (127.27.77.193:46279)
I20260806 00:57:20.369354 17403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f39e989849a44d58ff4d78bd4c57e36" candidate_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c"
I20260806 00:57:20.369426 17055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f39e989849a44d58ff4d78bd4c57e36" candidate_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a185f155050e4e458a419bf1e80b84e6"
I20260806 00:57:20.369469 17403 raft_consensus.cc:3037] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:57:20.369589 17055 raft_consensus.cc:3037] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:57:20.370652 17055 raft_consensus.cc:2454] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d3adbf2316b4d47a1465ee793d9e85c in term 1.
I20260806 00:57:20.370685 17403 raft_consensus.cc:2454] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d3adbf2316b4d47a1465ee793d9e85c in term 1.
I20260806 00:57:20.370941 17148 leader_election.cc:302] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d3adbf2316b4d47a1465ee793d9e85c, a185f155050e4e458a419bf1e80b84e6; no voters:
I20260806 00:57:20.371136 17494 raft_consensus.cc:2781] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:57:20.371421 17494 raft_consensus.cc:686] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [term 1 LEADER]: Becoming Leader. State: Replica: 0d3adbf2316b4d47a1465ee793d9e85c, State: Running, Role: LEADER
I20260806 00:57:20.371809 17494 consensus_queue.cc:237] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:20.375057 16847 catalog_manager.cc:5495] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194). New cstate: current_term: 1 leader_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } health_report { overall_health: UNKNOWN } } }
I20260806 00:57:20.397912 17486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.394211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58538 (local address 127.27.77.194:37553)
0806 00:57:20.394369 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:20.394374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:20.394392 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:57:20.394484 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:20.394487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:20.394566 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:20.394693 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:57:20.394952 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.395470 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.396350 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:20.396601 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:20.396720 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:20.397053 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:57:20.397455 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:57:20.397604 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:20.397669 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 00:57:20.397726 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:57:20.404091 17055 raft_consensus.cc:1261] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Refusing update from remote peer 0d3adbf2316b4d47a1465ee793d9e85c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:57:20.404094 17403 raft_consensus.cc:1261] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 1 FOLLOWER]: Refusing update from remote peer 0d3adbf2316b4d47a1465ee793d9e85c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:57:20.404943 17494 consensus_queue.cc:1035] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:57:20.405232 17499 consensus_queue.cc:1035] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:57:20.411088 17502 mvcc.cc:205] Tried to move back new op lower bound from 7315365234287390720 to 7315365234165579776. Current Snapshot: MvccSnapshot[applied={T|T < 7315365234287390720}]
I20260806 00:57:20.411764 17503 mvcc.cc:205] Tried to move back new op lower bound from 7315365234287390720 to 7315365234165579776. Current Snapshot: MvccSnapshot[applied={T|T < 7315365234287390720}]
I20260806 00:57:20.422273 16848 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:57:20.422492 16848 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:57:20.425141 16848 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Sending DeleteTablet for 3 replicas of tablet 6f39e989849a44d58ff4d78bd4c57e36
I20260806 00:57:20.425951 17021 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f39e989849a44d58ff4d78bd4c57e36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:57:20 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:41740
I20260806 00:57:20.425972 17208 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f39e989849a44d58ff4d78bd4c57e36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:57:20 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:58536
I20260806 00:57:20.426052 17378 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f39e989849a44d58ff4d78bd4c57e36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:57:20 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:37898
I20260806 00:57:20.426574 17519 tablet_replica.cc:330] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c: stopping tablet replica
I20260806 00:57:20.426579 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 16909
I20260806 00:57:20.436141 17519 raft_consensus.cc:2229] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:57:20.436669 17519 raft_consensus.cc:2258] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:57:20.437211 17518 tablet_replica.cc:330] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c: stopping tablet replica
I20260806 00:57:20.437594 17518 raft_consensus.cc:2229] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:57:20.438119 17518 raft_consensus.cc:2258] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:57:20.439726 17518 ts_tablet_manager.cc:1905] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:57:20.442653 17518 ts_tablet_manager.cc:1918] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:57:20.442809 17518 log.cc:1196] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/6f39e989849a44d58ff4d78bd4c57e36
W20260806 00:57:20.443734 16827 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:46279 (error 108)
I20260806 00:57:20.443842 17518 ts_tablet_manager.cc:1939] T 6f39e989849a44d58ff4d78bd4c57e36 P 0d3adbf2316b4d47a1465ee793d9e85c: Deleting consensus metadata
I20260806 00:57:20.444368 17519 ts_tablet_manager.cc:1905] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:57:20.445101 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 17120
I20260806 00:57:20.445067 16830 catalog_manager.cc:4841] TS 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553): tablet 6f39e989849a44d58ff4d78bd4c57e36 (table test-table [id=b1837c8680be42a2b794e6cf6ae200c9]) successfully deleted
I20260806 00:57:20.451917 17483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:20.451518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:46279 (local address 127.0.0.1:41756)
0806 00:57:20.451718 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:20.451805 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:46279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 00:57:20.452823 17519 ts_tablet_manager.cc:1918] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:57:20.452970 17519 log.cc:1196] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/6f39e989849a44d58ff4d78bd4c57e36
W20260806 00:57:20.452215 16827 catalog_manager.cc:4584] TS a185f155050e4e458a419bf1e80b84e6 (127.27.77.193:46279): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f39e989849a44d58ff4d78bd4c57e36: Network error: Client connection negotiation failed: client connection to 127.27.77.193:46279: connect: Connection refused (error 111)
I20260806 00:57:20.453346 17519 ts_tablet_manager.cc:1939] T 6f39e989849a44d58ff4d78bd4c57e36 P ef870f4eb07a49a2bbad1b1fd1f3d08c: Deleting consensus metadata
I20260806 00:57:20.454650 16830 catalog_manager.cc:4841] TS ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079): tablet 6f39e989849a44d58ff4d78bd4c57e36 (table test-table [id=b1837c8680be42a2b794e6cf6ae200c9]) successfully deleted
I20260806 00:57:20.458036 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 17300
I20260806 00:57:20.468992 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 16809
I20260806 00:57:40.481567 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38001
--webserver_interface=127.27.77.254
--webserver_port=46105
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:38001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:57:40.601311 17523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:40.601621 17523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:40.601718 17523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:40.605782 17523 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 00:57:40.605861 17523 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 00:57:40.605890 17523 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 00:57:40.605922 17523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:57:40.605962 17523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:40.606014 17523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:40.606065 17523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:57:40.606102 17523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:57:40.611567 17523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:38001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=46105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:57:40.612731 17523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:57:40.613771 17523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:57:40.623183 17523 server_base.cc:1034] running on GCE node
W20260806 00:57:40.623126 17532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:40.623247 17535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:40.623127 17533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:40.624044 17523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:57:40.625190 17523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:57:40.626430 17523 hybrid_clock.cc:648] HybridClock initialized: now 1785977860626403 us; error 39 us; skew 500 ppm
Aug 06 00:57:40 dist-test-slave-sb2z krb5kdc[16063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977860, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:57:40.629488 17523 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:57:40.630386 17523 webserver.cc:460] Webserver started at http://127.27.77.254:46105/ using document root <none> and password file <none>
I20260806 00:57:40.630905 17523 fs_manager.cc:359] Metadata directory not provided
I20260806 00:57:40.630965 17523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:57:40.634838 17523 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:57:40.637127 17543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:40.638461 17523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260806 00:57:40.638576 17523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "6a0f517681684136a90430c795ded7cc"
format_stamp: "Formatted at 2026-08-06 00:56:58 on dist-test-slave-sb2z"
server_key: "2e466672b1492ea07ca2ef943c6068d5"
server_key_iv: "bf57644131e95e3cf9b73799196fa4d8"
server_key_version: "encryptionkey@0"
I20260806 00:57:40.639125 17523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:57:40.664891 17523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:57:40.668368 17523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:57:40.668538 17523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:57:40.676220 17523 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:38001
I20260806 00:57:40.676538 17608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:38001 every 8 connection(s)
I20260806 00:57:40.677362 17523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:57:40.680366 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 17523
I20260806 00:57:40.680927 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:46279
--local_ip_for_outbound_sockets=127.27.77.193
--tserver_master_addrs=127.27.77.254:38001
--webserver_port=43965
--webserver_interface=127.27.77.193
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:57:40.682354 17609 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:57:40.684476 17609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Bootstrap starting.
I20260806 00:57:40.703336 17609 log.cc:824] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Log is configured to *not* fsync() on all Append() calls
I20260806 00:57:40.714542 17609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:57:40.714951 17609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Bootstrap complete.
I20260806 00:57:40.719143 17609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } }
I20260806 00:57:40.719518 17609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a0f517681684136a90430c795ded7cc, State: Initialized, Role: FOLLOWER
I20260806 00:57:40.720029 17609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } }
I20260806 00:57:40.720147 17609 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:57:40.720234 17609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:57:40.720346 17609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 2 FOLLOWER]: Advancing to term 3
I20260806 00:57:40.722239 17609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } }
I20260806 00:57:40.722579 17609 leader_election.cc:302] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a0f517681684136a90430c795ded7cc; no voters:
I20260806 00:57:40.722922 17609 leader_election.cc:288] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 00:57:40.723007 17614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 3 FOLLOWER]: Leader election won for term 3
I20260806 00:57:40.723196 17614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [term 3 LEADER]: Becoming Leader. State: Replica: 6a0f517681684136a90430c795ded7cc, State: Running, Role: LEADER
I20260806 00:57:40.723577 17614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } }
I20260806 00:57:40.724216 17609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:57:40.725010 17617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a0f517681684136a90430c795ded7cc. Latest consensus state: current_term: 3 leader_uuid: "6a0f517681684136a90430c795ded7cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } } }
I20260806 00:57:40.725392 17617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [sys.catalog]: This master's current role is: LEADER
I20260806 00:57:40.725291 17615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6a0f517681684136a90430c795ded7cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0f517681684136a90430c795ded7cc" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38001 } } }
I20260806 00:57:40.725770 17615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc [sys.catalog]: This master's current role is: LEADER
I20260806 00:57:40.726135 17626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:57:40.731503 17626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:57:40.732154 17626 catalog_manager.cc:1259] Loaded cluster ID: c159e7cb46da44bf9f56636be35754f6
I20260806 00:57:40.732218 17626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:57:40.735265 17626 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:57:40.736119 17626 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Loaded TSK: 0
I20260806 00:57:40.736209 17626 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6a0f517681684136a90430c795ded7cc: Loaded TSK: 1
F20260806 00:57:40.736948 17626 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1e193b29ad google::LogMessage::Fail() at ??:0
@ 0x7f1e193b588a google::LogMessage::SendToLog() at ??:0
@ 0x7f1e193b24de google::LogMessage::Flush() at ??:0
@ 0x7f1e193b3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1e1c07618d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f1e1c06ef0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f1e23d177bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f1e23cf199c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f1e23cf2178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f1e23d33df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f1e23d5282b std::function<>::operator()() at ??:0
@ 0x7f1e23cf1b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f1e23cf2f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1e23cef579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f1e23d33640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f1e1b6043d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1e1b604c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f1e1b6063f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f1e1b5f5512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1e1cc396db start_thread at ??:0
@ 0x7f1e17eb061f clone at ??:0
W20260806 00:57:40.842573 17611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:40.842969 17611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:40.843048 17611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:40.849584 17611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:40.849680 17611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:40.849784 17611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:57:40.858196 17611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:46279
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=43965
--tserver_master_addrs=127.27.77.254:38001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:57:40.859951 17611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:57:40.861471 17611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:57:40.873744 17673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:40.873818 17670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:40.873744 17669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:40.874102 17611 server_base.cc:1034] running on GCE node
I20260806 00:57:40.874580 17611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:57:40.875411 17611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:57:40.876616 17611 hybrid_clock.cc:648] HybridClock initialized: now 1785977860876588 us; error 38 us; skew 500 ppm
Aug 06 00:57:40 dist-test-slave-sb2z krb5kdc[16063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977860, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:57:40.880409 17611 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:57:40.881714 17611 webserver.cc:460] Webserver started at http://127.27.77.193:43965/ using document root <none> and password file <none>
I20260806 00:57:40.882550 17611 fs_manager.cc:359] Metadata directory not provided
I20260806 00:57:40.882653 17611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:57:40.890805 17611 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.004s
I20260806 00:57:40.893702 17680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:40.895169 17611 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:57:40.895324 17611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "a185f155050e4e458a419bf1e80b84e6"
format_stamp: "Formatted at 2026-08-06 00:56:58 on dist-test-slave-sb2z"
server_key: "1da3955ee1ba2127471670fb38cdfa3d"
server_key_iv: "a86a44c4dae836089752fc4c615c1e3d"
server_key_version: "encryptionkey@0"
I20260806 00:57:40.895977 17611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:57:40.911686 17611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:57:40.915019 17611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:57:40.915237 17611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:57:40.915923 17611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:57:40.917063 17690 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 00:57:40.922612 17611 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 00:57:40.922703 17611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260806 00:57:40.922796 17611 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 00:57:40.924609 17611 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 00:57:40.924680 17611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 00:57:40.924998 17690 tablet_bootstrap.cc:492] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6: Bootstrap starting.
I20260806 00:57:40.935901 17611 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:46279
I20260806 00:57:40.936120 17844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:46279 every 8 connection(s)
I20260806 00:57:40.936928 17611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:57:40.941131 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 17611
I20260806 00:57:40.941619 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:37553
--local_ip_for_outbound_sockets=127.27.77.194
--tserver_master_addrs=127.27.77.254:38001
--webserver_port=38537
--webserver_interface=127.27.77.194
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:57:40.950995 17690 log.cc:824] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6: Log is configured to *not* fsync() on all Append() calls
I20260806 00:57:40.958195 17690 tablet_bootstrap.cc:492] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:57:40.958617 17690 tablet_bootstrap.cc:492] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6: Bootstrap complete.
I20260806 00:57:40.960098 17690 ts_tablet_manager.cc:1397] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6: Time spent bootstrapping tablet: real 0.035s user 0.019s sys 0.013s
I20260806 00:57:40.962607 17690 raft_consensus.cc:348] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:40.963064 17690 raft_consensus.cc:729] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a185f155050e4e458a419bf1e80b84e6, State: Initialized, Role: FOLLOWER
I20260806 00:57:40.963615 17690 consensus_queue.cc:260] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:40.964612 17690 ts_tablet_manager.cc:1428] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260806 00:57:41.029506 17850 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:57:40.939109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:52401)
0806 00:57:40.939639 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:40.939695 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:40.940553 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:41.028703 (+ 88150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":150,"threads_started":1}
W20260806 00:57:41.030045 17846 heartbeater.cc:644] Failed to heartbeat to 127.27.77.254:38001 (0 consecutive failures): Network error: Failed to ping master at 127.27.77.254:38001: Client connection negotiation failed: client connection to 127.27.77.254:38001: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 00:57:41.030848 17850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:41.030586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:45935)
0806 00:57:41.030725 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:41.030791 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 00:57:41.031666 17850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:41.031397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:44795)
0806 00:57:41.031560 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:41.031616 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 00:57:41.031847 17846 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:57:41.084892 17851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:41.085188 17851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:41.085269 17851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:41.089264 17851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:41.089354 17851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:41.089443 17851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:57:41.094798 17851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:37553
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=38537
--tserver_master_addrs=127.27.77.254:38001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:57:41.095963 17851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:57:41.097014 17851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:57:41.106976 17861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:41.107076 17859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:41.107189 17851 server_base.cc:1034] running on GCE node
W20260806 00:57:41.107014 17866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:41.107761 17851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:57:41.108455 17851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:57:41.109660 17851 hybrid_clock.cc:648] HybridClock initialized: now 1785977861109630 us; error 45 us; skew 500 ppm
Aug 06 00:57:41 dist-test-slave-sb2z krb5kdc[16063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977861, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:57:41.112777 17851 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:57:41.113873 17851 webserver.cc:460] Webserver started at http://127.27.77.194:38537/ using document root <none> and password file <none>
I20260806 00:57:41.114544 17851 fs_manager.cc:359] Metadata directory not provided
I20260806 00:57:41.114643 17851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:57:41.118742 17851 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:57:41.121035 17875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:41.122406 17851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 00:57:41.122587 17851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "0d3adbf2316b4d47a1465ee793d9e85c"
format_stamp: "Formatted at 2026-08-06 00:56:58 on dist-test-slave-sb2z"
server_key: "aa47892fa16621b2df76b0e260d86c76"
server_key_iv: "ab16397e0741c817d618f3918c502ff0"
server_key_version: "encryptionkey@0"
I20260806 00:57:41.123212 17851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:57:41.135869 17851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:57:41.139063 17851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:57:41.139285 17851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:57:41.139931 17851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:57:41.140799 17851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:57:41.140869 17851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:41.140961 17851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:57:41.141002 17851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:41.150517 17851 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:37553
I20260806 00:57:41.150547 18028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:37553 every 8 connection(s)
I20260806 00:57:41.151638 17851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:57:41.155483 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 17851
I20260806 00:57:41.155692 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:41.153926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:52943)
0806 00:57:41.154677 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:41.154790 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":106,"threads_started":1}
I20260806 00:57:41.156050 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:36079
--local_ip_for_outbound_sockets=127.27.77.195
--tserver_master_addrs=127.27.77.254:38001
--webserver_port=43301
--webserver_interface=127.27.77.195
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:57:41.156394 18029 heartbeater.cc:644] Failed to heartbeat to 127.27.77.254:38001 (0 consecutive failures): Network error: Failed to ping master at 127.27.77.254:38001: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
I20260806 00:57:41.157091 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:41.156835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:42889)
0806 00:57:41.156976 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:41.157034 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 00:57:41.157732 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:41.157519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:38307)
0806 00:57:41.157631 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:41.157680 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 00:57:41.157894 18029 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:57:41.270087 18033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:41.270350 18033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:41.270395 18033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:41.274478 18033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:41.274546 18033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:41.274607 18033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:57:41.280436 18033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:36079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=43301
--tserver_master_addrs=127.27.77.254:38001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:57:41.281579 18033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:57:41.282635 18033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:57:41.293198 18041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:41.293162 18040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:41.293239 18033 server_base.cc:1034] running on GCE node
W20260806 00:57:41.293183 18043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:41.293951 18033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:57:41.294615 18033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:57:41.295760 18033 hybrid_clock.cc:648] HybridClock initialized: now 1785977861295744 us; error 28 us; skew 500 ppm
Aug 06 00:57:41 dist-test-slave-sb2z krb5kdc[16063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977861, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:57:41.298681 18033 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:57:41.299705 18033 webserver.cc:460] Webserver started at http://127.27.77.195:43301/ using document root <none> and password file <none>
I20260806 00:57:41.300318 18033 fs_manager.cc:359] Metadata directory not provided
I20260806 00:57:41.300413 18033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:57:41.304600 18033 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 00:57:41.306725 18051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:41.308157 18033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:57:41.308300 18033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c"
format_stamp: "Formatted at 2026-08-06 00:56:59 on dist-test-slave-sb2z"
server_key: "49c3870aa3e743f9166b3c1937e68a1d"
server_key_iv: "b99df649d2eaecc14ddbf37c26dcfe83"
server_key_version: "encryptionkey@0"
I20260806 00:57:41.308857 18033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:57:41.322184 18033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:57:41.326088 18033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:57:41.326411 18033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:57:41.328491 18033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:57:41.329999 18033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:57:41.330082 18033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:41.330154 18033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:57:41.330188 18033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:41.343855 18033 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:36079
I20260806 00:57:41.343873 18211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:36079 every 8 connection(s)
I20260806 00:57:41.345082 18033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:57:41.350358 18214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:41.347996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:52073)
0806 00:57:41.348999 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:41.349189 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":171,"threads_started":1}
W20260806 00:57:41.350955 18212 heartbeater.cc:644] Failed to heartbeat to 127.27.77.254:38001 (0 consecutive failures): Network error: Failed to ping master at 127.27.77.254:38001: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
I20260806 00:57:41.351755 18214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:41.351424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:53671)
0806 00:57:41.351611 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:41.351681 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 00:57:41.352389 18214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:41.352186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:42317)
0806 00:57:41.352285 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:41.352333 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 00:57:41.352574 18212 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 00:57:41.354578 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 18033
I20260806 00:57:42.032963 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:42.032467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:57273)
0806 00:57:42.032771 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:42.032838 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":83,"threads_started":1}
I20260806 00:57:42.158910 18217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:42.158482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:57107)
0806 00:57:42.158749 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:42.158807 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":78,"threads_started":1}
I20260806 00:57:42.315968 18218 raft_consensus.cc:482] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:57:42.316176 18218 raft_consensus.cc:504] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:42.317377 18218 leader_election.cc:288] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553), ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079)
I20260806 00:57:42.353578 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:42.353151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:56687)
0806 00:57:42.353406 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:42.353462 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":79,"threads_started":1}
I20260806 00:57:43.034394 18230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:43.033923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:51293)
0806 00:57:43.034213 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:43.034283 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":83,"threads_started":1}
I20260806 00:57:43.160537 18234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:43.159990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:45723)
0806 00:57:43.160341 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:43.160405 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":96,"threads_started":1}
W20260806 00:57:43.286499 18216 sasl_common.cc:323] Time spent creating new SASL kudu client: real 0.962s user 0.000s sys 0.000s
W20260806 00:57:43.286511 18226 sasl_common.cc:323] Time spent creating new SASL kudu client: real 0.962s user 0.000s sys 0.000s
Aug 06 00:57:43 dist-test-slave-sb2z krb5kdc[16063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977860, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.194@KRBTEST.COM
Aug 06 00:57:43 dist-test-slave-sb2z krb5kdc[16063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977860, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.195@KRBTEST.COM
I20260806 00:57:43.295120 18226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:42.317998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:36079 (local address 127.27.77.193:41791)
0806 00:57:42.318395 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:42.318412 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:42.318531 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:42.320013 (+ 1482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:42.320016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:42.320586 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:57:42.321075 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:42.321092 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:42.322807 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:42.322811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:42.323805 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:42.323815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:42.324555 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:42.324559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:42.324774 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:43.286735 (+961961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:57:43.286768 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:57:43.289027 (+ 2259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:57:43.291564 (+ 2537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:43.291570 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:43.291573 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:43.291862 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:43.292330 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:43.292334 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:43.292337 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:43.292603 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:43.293721 (+ 1118us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:57:43.293728 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:57:43.294151 (+ 423us) client_negotiation.cc:769] Sending connection context
0806 00:57:43.294398 (+ 247us) client_negotiation.cc:240] Negotiation successful
0806 00:57:43.294949 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":313,"spinlock_wait_cycles":50304,"thread_start_us":93,"threads_started":1}
I20260806 00:57:43.295167 18224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:42.317839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:43593 (local address 127.27.77.194:37553)
0806 00:57:42.318289 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:42.318325 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:42.318368 (+ 43us) server_negotiation.cc:407] Connection header received
0806 00:57:42.319113 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:42.319120 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:42.319475 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:42.319873 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:42.321408 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:42.322641 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:42.323921 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:42.324386 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:42.324578 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:43.289097 (+964519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:43.289109 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:43.289133 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:57:43.289187 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:43.291245 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:43.291981 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:43.291989 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:43.291995 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:43.292171 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:43.292781 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:43.292786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:43.292788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:43.293483 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:43.293646 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:43.294597 (+ 951us) server_negotiation.cc:299] Negotiation successful
0806 00:57:43.294957 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":82,"threads_started":1}
I20260806 00:57:43.295163 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:42.318214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:41791 (local address 127.27.77.195:36079)
0806 00:57:42.318672 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:42.318686 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:42.318727 (+ 41us) server_negotiation.cc:407] Connection header received
0806 00:57:42.319376 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:42.319382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:42.319788 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:42.320083 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:42.321260 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:42.322669 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:42.323933 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:42.324426 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:42.324608 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:43.289343 (+964735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:43.289350 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:43.289363 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:57:43.289402 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:43.291395 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:43.292026 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:43.292031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:43.292037 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:43.292219 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:43.292781 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:43.292786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:43.292788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:43.293483 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:43.293646 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:43.294602 (+ 956us) server_negotiation.cc:299] Negotiation successful
0806 00:57:43.294954 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":122,"threads_started":1}
I20260806 00:57:43.295120 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:42.317683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:37553 (local address 127.27.77.193:43593)
0806 00:57:42.317835 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:42.317853 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:42.317972 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:42.319927 (+ 1955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:42.319934 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:42.320553 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:57:42.321216 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:42.321228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:42.322798 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:42.322801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:42.323796 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:42.323806 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:42.324523 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:42.324526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:42.324773 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:43.286680 (+961907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:57:43.286718 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:57:43.288654 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:57:43.291394 (+ 2740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:43.291402 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:43.291417 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:43.291831 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:43.292330 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:43.292334 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:43.292337 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:43.292603 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:43.293721 (+ 1118us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:57:43.293728 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:57:43.294151 (+ 423us) client_negotiation.cc:769] Sending connection context
0806 00:57:43.294398 (+ 247us) client_negotiation.cc:240] Negotiation successful
0806 00:57:43.294878 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":309,"spinlock_wait_cycles":12160}
W20260806 00:57:43.298287 17684 leader_election.cc:341] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553): Not found: Tablet not found: 6f39e989849a44d58ff4d78bd4c57e36
W20260806 00:57:43.299006 17684 leader_election.cc:341] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079): Not found: Tablet not found: 6f39e989849a44d58ff4d78bd4c57e36
I20260806 00:57:43.299115 17684 leader_election.cc:302] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a185f155050e4e458a419bf1e80b84e6; no voters: 0d3adbf2316b4d47a1465ee793d9e85c, ef870f4eb07a49a2bbad1b1fd1f3d08c
I20260806 00:57:43.299626 18235 raft_consensus.cc:2726] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:57:43.354820 18236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:43.354363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:60593)
0806 00:57:43.354646 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:43.354704 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":108,"threads_started":1}
I20260806 00:57:44.035808 18237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:44.035274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:50623)
0806 00:57:44.035634 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:44.035695 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":115,"threads_started":1}
I20260806 00:57:44.161823 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:44.161355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:46791)
0806 00:57:44.161649 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:44.161710 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":96,"threads_started":1}
I20260806 00:57:44.356187 18239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:44.355738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:38319)
0806 00:57:44.356016 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:44.356085 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":78,"threads_started":1}
I20260806 00:57:44.938624 18240 raft_consensus.cc:482] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:57:44.938759 18240 raft_consensus.cc:504] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:44.939183 18240 leader_election.cc:288] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553), ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079)
W20260806 00:57:44.940282 17684 leader_election.cc:341] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553): Not found: Tablet not found: 6f39e989849a44d58ff4d78bd4c57e36
W20260806 00:57:44.940454 17684 leader_election.cc:341] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079): Not found: Tablet not found: 6f39e989849a44d58ff4d78bd4c57e36
I20260806 00:57:44.940538 17684 leader_election.cc:302] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a185f155050e4e458a419bf1e80b84e6; no voters: 0d3adbf2316b4d47a1465ee793d9e85c, ef870f4eb07a49a2bbad1b1fd1f3d08c
I20260806 00:57:44.940713 18240 raft_consensus.cc:2726] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:57:45.037168 18241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:45.036701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:41601)
0806 00:57:45.036998 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:45.037061 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":85,"threads_started":1}
I20260806 00:57:45.163163 18242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:45.162711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:41517)
0806 00:57:45.162986 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:45.163049 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":81,"threads_started":1}
I20260806 00:57:45.357430 18243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:45.356995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:46075)
0806 00:57:45.357258 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:45.357317 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":74,"threads_started":1}
I20260806 00:57:46.038793 18244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:46.038304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:50715)
0806 00:57:46.038630 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:46.038692 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":76,"threads_started":1}
I20260806 00:57:46.164535 18246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:46.164044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:44087)
0806 00:57:46.164367 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:46.164430 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":106,"threads_started":1}
I20260806 00:57:46.359059 18248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:46.358494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:42645)
0806 00:57:46.358819 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:46.358919 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":130,"threads_started":1}
I20260806 00:57:47.040410 18249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:47.039868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:34245)
0806 00:57:47.040241 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:47.040307 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":149,"threads_started":1}
I20260806 00:57:47.165807 18251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:47.165372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:45265)
0806 00:57:47.165642 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:47.165707 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":76,"threads_started":1}
I20260806 00:57:47.360643 18252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:47.360153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:35937)
0806 00:57:47.360460 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:47.360529 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":91,"threads_started":1}
I20260806 00:57:47.729063 18253 raft_consensus.cc:482] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:57:47.729256 18253 raft_consensus.cc:504] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:47.729803 18253 leader_election.cc:288] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553), ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079)
W20260806 00:57:47.730800 17684 leader_election.cc:341] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079): Not found: Tablet not found: 6f39e989849a44d58ff4d78bd4c57e36
W20260806 00:57:47.730926 17684 leader_election.cc:341] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553): Not found: Tablet not found: 6f39e989849a44d58ff4d78bd4c57e36
I20260806 00:57:47.731004 17684 leader_election.cc:302] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a185f155050e4e458a419bf1e80b84e6; no voters: 0d3adbf2316b4d47a1465ee793d9e85c, ef870f4eb07a49a2bbad1b1fd1f3d08c
I20260806 00:57:47.731158 18253 raft_consensus.cc:2726] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:57:48.042121 18254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:48.041506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:48019)
0806 00:57:48.041903 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:48.042008 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":107,"threads_started":1}
I20260806 00:57:48.167263 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:48.166741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:60539)
0806 00:57:48.167088 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:48.167156 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":78,"threads_started":1}
I20260806 00:57:48.362252 18257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:48.361629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:60483)
0806 00:57:48.362062 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:48.362136 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":85,"threads_started":1}
I20260806 00:57:49.043836 18260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:49.043235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:46295)
0806 00:57:49.043645 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:49.043713 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":90,"threads_started":1}
I20260806 00:57:49.168613 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:49.168126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:51793)
0806 00:57:49.168438 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:49.168516 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":82,"threads_started":1}
I20260806 00:57:49.363858 18263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:49.363278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:46579)
0806 00:57:49.363679 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:49.363750 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":97,"threads_started":1}
I20260806 00:57:49.682251 18264 raft_consensus.cc:482] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:57:49.682421 18264 raft_consensus.cc:504] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:49.682915 18264 leader_election.cc:288] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553), ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079)
W20260806 00:57:49.683904 17684 leader_election.cc:341] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553): Not found: Tablet not found: 6f39e989849a44d58ff4d78bd4c57e36
W20260806 00:57:49.684037 17684 leader_election.cc:341] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079): Not found: Tablet not found: 6f39e989849a44d58ff4d78bd4c57e36
I20260806 00:57:49.684069 17684 leader_election.cc:302] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a185f155050e4e458a419bf1e80b84e6; no voters: 0d3adbf2316b4d47a1465ee793d9e85c, ef870f4eb07a49a2bbad1b1fd1f3d08c
I20260806 00:57:49.684213 18264 raft_consensus.cc:2726] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:57:50.045364 18265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:50.044798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:48315)
0806 00:57:50.045177 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:50.045245 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":107,"threads_started":1}
I20260806 00:57:50.170039 18266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:50.169550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:51459)
0806 00:57:50.169871 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:50.169935 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":128,"threads_started":1}
I20260806 00:57:50.365298 18267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:50.364778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:60141)
0806 00:57:50.365126 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:50.365193 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":89,"threads_started":1}
I20260806 00:57:51.046789 18268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:51.046297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:52983)
0806 00:57:51.046624 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:51.046687 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":81,"threads_started":1}
I20260806 00:57:51.171636 18269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:51.171084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:54701)
0806 00:57:51.171377 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:51.171442 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":85,"threads_started":1}
I20260806 00:57:51.366743 18270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:51.366265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:33503)
0806 00:57:51.366576 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:51.366643 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":88,"threads_started":1}
I20260806 00:57:52.048247 18271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:52.047793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:49849)
0806 00:57:52.048074 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:52.048135 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":90,"threads_started":1}
I20260806 00:57:52.172948 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:52.172488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:54631)
0806 00:57:52.172777 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:52.172843 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":81,"threads_started":1}
I20260806 00:57:52.368288 18275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:52.367810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:38543)
0806 00:57:52.368137 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:52.368188 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":102,"threads_started":1}
I20260806 00:57:53.049633 18276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:53.049140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:48693)
0806 00:57:53.049462 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:53.049527 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":86,"threads_started":1}
I20260806 00:57:53.174279 18278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:53.173779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:36523)
0806 00:57:53.174079 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:53.174141 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":103,"threads_started":1}
I20260806 00:57:53.369813 18279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:53.369264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:54119)
0806 00:57:53.369620 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:53.369694 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":91,"threads_started":1}
I20260806 00:57:54.051234 18280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:54.050616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:37021)
0806 00:57:54.050981 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:54.051120 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":114,"threads_started":1}
I20260806 00:57:54.175599 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:54.175143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:41295)
0806 00:57:54.175409 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:54.175476 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":82,"threads_started":1}
I20260806 00:57:54.371398 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:54.370868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:43979)
0806 00:57:54.371222 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:54.371291 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":108,"threads_started":1}
I20260806 00:57:55.052632 18283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:55.052124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:33509)
0806 00:57:55.052453 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:55.052524 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":84,"threads_started":1}
I20260806 00:57:55.177121 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:55.176568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:45939)
0806 00:57:55.176908 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:55.176976 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":95,"threads_started":1}
I20260806 00:57:55.373047 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:55.372523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:53801)
0806 00:57:55.372865 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:55.372938 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":119,"threads_started":1}
I20260806 00:57:55.896692 18286 raft_consensus.cc:482] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:57:55.896869 18286 raft_consensus.cc:504] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d3adbf2316b4d47a1465ee793d9e85c" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37553 } } peers { permanent_uuid: "ef870f4eb07a49a2bbad1b1fd1f3d08c" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 36079 } } peers { permanent_uuid: "a185f155050e4e458a419bf1e80b84e6" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 46279 } }
I20260806 00:57:55.897311 18286 leader_election.cc:288] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553), ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079)
W20260806 00:57:55.898285 17684 leader_election.cc:341] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d3adbf2316b4d47a1465ee793d9e85c (127.27.77.194:37553): Not found: Tablet not found: 6f39e989849a44d58ff4d78bd4c57e36
W20260806 00:57:55.898468 17684 leader_election.cc:341] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ef870f4eb07a49a2bbad1b1fd1f3d08c (127.27.77.195:36079): Not found: Tablet not found: 6f39e989849a44d58ff4d78bd4c57e36
I20260806 00:57:55.898535 17684 leader_election.cc:302] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a185f155050e4e458a419bf1e80b84e6; no voters: 0d3adbf2316b4d47a1465ee793d9e85c, ef870f4eb07a49a2bbad1b1fd1f3d08c
I20260806 00:57:55.898726 18286 raft_consensus.cc:2726] T 6f39e989849a44d58ff4d78bd4c57e36 P a185f155050e4e458a419bf1e80b84e6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:57:56.054205 18287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:56.053719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.193:55121)
0806 00:57:56.054030 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:56.054094 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":81,"threads_started":1}
I20260806 00:57:56.178680 18288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:56.178184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.194:54727)
0806 00:57:56.178507 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:56.178572 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":99,"threads_started":1}
I20260806 00:57:56.362290 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 17611
I20260806 00:57:56.374809 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 17851
I20260806 00:57:56.374859 18289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:56.374290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:38001 (local address 127.27.77.195:38141)
0806 00:57:56.374711 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:56.374762 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.254:38001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":88,"threads_started":1}
I20260806 00:57:56.386534 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 18033
2026-08-06T00:57:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59632 ms)
[----------] 30 tests from SecurityITest (257330 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:57:56 dist-test-slave-sb2z krb5kdc[18292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:57:56 dist-test-slave-sb2z krb5kdc[18292](info): set up 2 sockets
Aug 06 00:57:56 dist-test-slave-sb2z krb5kdc[18292](info): commencing operation
krb5kdc: starting...
W20260806 00:57:57.638859 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:57:57 dist-test-slave-sb2z krb5kdc[18292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:57:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:57:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:57:57.783190 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:44551
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:44551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:57:57.900395 18312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:57.900718 18312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:57.900792 18312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:57.904923 18312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:57:57.905018 18312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:57.905056 18312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:57.905095 18312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:57:57.905130 18312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:57:57.910653 18312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:44551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:44551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:57:57.912101 18312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:57:57.913300 18312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:57:57.923600 18312 server_base.cc:1034] running on GCE node
W20260806 00:57:57.923542 18322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:57.923518 18318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:57.923519 18319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:57.924595 18312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:57:57.925758 18312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:57:57.926939 18312 hybrid_clock.cc:648] HybridClock initialized: now 1785977877926926 us; error 31 us; skew 500 ppm
Aug 06 00:57:57 dist-test-slave-sb2z krb5kdc[18292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977877, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:57:57.930128 18312 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:57:57.931097 18312 webserver.cc:460] Webserver started at http://127.27.77.254:33833/ using document root <none> and password file <none>
I20260806 00:57:57.931733 18312 fs_manager.cc:359] Metadata directory not provided
I20260806 00:57:57.931807 18312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:57:57.931988 18312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:57:57.934051 18312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "c3f054073d464242964a1c6bc62a21be"
format_stamp: "Formatted at 2026-08-06 00:57:57 on dist-test-slave-sb2z"
server_key: "ea61a58d3eaf271a9144be2b7b262189"
server_key_iv: "7d66ee820c827733cf1f1c3448d68cf8"
server_key_version: "encryptionkey@0"
I20260806 00:57:57.934582 18312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "c3f054073d464242964a1c6bc62a21be"
format_stamp: "Formatted at 2026-08-06 00:57:57 on dist-test-slave-sb2z"
server_key: "ea61a58d3eaf271a9144be2b7b262189"
server_key_iv: "7d66ee820c827733cf1f1c3448d68cf8"
server_key_version: "encryptionkey@0"
I20260806 00:57:57.938683 18312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 00:57:57.941316 18331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:57.942672 18312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 00:57:57.942843 18312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "c3f054073d464242964a1c6bc62a21be"
format_stamp: "Formatted at 2026-08-06 00:57:57 on dist-test-slave-sb2z"
server_key: "ea61a58d3eaf271a9144be2b7b262189"
server_key_iv: "7d66ee820c827733cf1f1c3448d68cf8"
server_key_version: "encryptionkey@0"
I20260806 00:57:57.942974 18312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:57:57.968438 18312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:57:57.972203 18312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:57:57.972432 18312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:57:57.980413 18312 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:44551
I20260806 00:57:57.980433 18399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:44551 every 8 connection(s)
I20260806 00:57:57.981453 18312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:57:57.981961 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 18312
I20260806 00:57:57.982177 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:57:57.982528 27959 external_mini_cluster.cc:1428] Setting key c04b8fa714850d30bb6e9401510c0ba3
I20260806 00:57:57.984812 18400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:57:57.992028 18400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be: Bootstrap starting.
Aug 06 00:57:57 dist-test-slave-sb2z krb5kdc[18292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:57:57.995328 18400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be: Neither blocks nor log segments found. Creating new log.
I20260806 00:57:57.996352 18400 log.cc:824] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be: Log is configured to *not* fsync() on all Append() calls
I20260806 00:57:57.998618 18400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be: No bootstrap required, opened a new log
I20260806 00:57:57.999444 18404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:57.984139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42768 (local address 127.27.77.254:44551)
0806 00:57:57.984605 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:57.984626 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:57.984671 (+ 45us) server_negotiation.cc:407] Connection header received
0806 00:57:57.985517 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:57.985541 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:57.985967 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:57.986340 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:57.987066 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:57.988658 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:57.989806 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:57.990523 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:57.990709 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:57.993572 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:57.993580 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:57.993599 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:57:57.993641 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:57.996018 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:57.996647 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:57.996653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:57.996659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:57.996794 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:57.997145 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:57.997148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:57.997150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:57.997620 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:57.997860 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:57.998247 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 00:57:57.998601 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":129,"threads_started":1}
I20260806 00:57:58.002966 18400 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f054073d464242964a1c6bc62a21be" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 44551 } }
I20260806 00:57:58.003196 18400 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:57:58.003273 18400 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f054073d464242964a1c6bc62a21be, State: Initialized, Role: FOLLOWER
I20260806 00:57:58.003775 18400 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f054073d464242964a1c6bc62a21be" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 44551 } }
I20260806 00:57:58.003957 18400 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:57:58.004035 18400 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:57:58.004143 18400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:57:58.005129 18400 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f054073d464242964a1c6bc62a21be" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 44551 } }
I20260806 00:57:58.005489 18400 leader_election.cc:302] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3f054073d464242964a1c6bc62a21be; no voters:
I20260806 00:57:58.005820 18400 leader_election.cc:288] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:57:58.006058 18406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:57:58.006381 18406 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [term 1 LEADER]: Becoming Leader. State: Replica: c3f054073d464242964a1c6bc62a21be, State: Running, Role: LEADER
I20260806 00:57:58.006759 18406 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f054073d464242964a1c6bc62a21be" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 44551 } }
I20260806 00:57:58.007177 18400 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:57:58.008577 18408 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3f054073d464242964a1c6bc62a21be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f054073d464242964a1c6bc62a21be" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 44551 } } }
I20260806 00:57:58.008725 18408 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [sys.catalog]: This master's current role is: LEADER
I20260806 00:57:58.009136 18409 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3f054073d464242964a1c6bc62a21be. Latest consensus state: current_term: 1 leader_uuid: "c3f054073d464242964a1c6bc62a21be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f054073d464242964a1c6bc62a21be" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 44551 } } }
I20260806 00:57:58.009305 18409 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be [sys.catalog]: This master's current role is: LEADER
W20260806 00:57:58.012354 18429 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:57:58.012465 18429 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:57:58.012545 18421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:57:58.013316 18421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:57:58.018950 18421 catalog_manager.cc:1347] Generated new cluster ID: 05507e4716b44ccfae3c71f4971926ef
I20260806 00:57:58.019029 18421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:57:58.042544 18421 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:57:58.043752 18421 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:57:58.047595 18421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be: Generated new TSK 0
I20260806 00:57:58.048331 18421 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:57:58.091811 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:44551
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:57:58.215140 18438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:58.215394 18438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:58.215440 18438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:58.219345 18438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:58.219403 18438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:58.219460 18438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:57:58.224651 18438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:44551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:57:58.225664 18438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:57:58.226629 18438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:57:58.236281 18446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:58.236300 18448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:58.236403 18438 server_base.cc:1034] running on GCE node
W20260806 00:57:58.236308 18445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:58.237134 18438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:57:58.237824 18438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:57:58.239032 18438 hybrid_clock.cc:648] HybridClock initialized: now 1785977878239014 us; error 34 us; skew 500 ppm
Aug 06 00:57:58 dist-test-slave-sb2z krb5kdc[18292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977878, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:57:58.242285 18438 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:57:58.243181 18438 webserver.cc:460] Webserver started at http://127.27.77.193:33323/ using document root <none> and password file <none>
I20260806 00:57:58.243825 18438 fs_manager.cc:359] Metadata directory not provided
I20260806 00:57:58.243922 18438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:57:58.244163 18438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:57:58.246201 18438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "53bd8ab29cfc491faa405848cbb1bf37"
format_stamp: "Formatted at 2026-08-06 00:57:58 on dist-test-slave-sb2z"
server_key: "8cf4d1c9d378666faed250d3db5f24f1"
server_key_iv: "f10e75e5fff167a439b72456edadc616"
server_key_version: "encryptionkey@0"
I20260806 00:57:58.246843 18438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "53bd8ab29cfc491faa405848cbb1bf37"
format_stamp: "Formatted at 2026-08-06 00:57:58 on dist-test-slave-sb2z"
server_key: "8cf4d1c9d378666faed250d3db5f24f1"
server_key_iv: "f10e75e5fff167a439b72456edadc616"
server_key_version: "encryptionkey@0"
I20260806 00:57:58.250717 18438 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:57:58.253312 18455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:58.254597 18438 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:57:58.254750 18438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "53bd8ab29cfc491faa405848cbb1bf37"
format_stamp: "Formatted at 2026-08-06 00:57:58 on dist-test-slave-sb2z"
server_key: "8cf4d1c9d378666faed250d3db5f24f1"
server_key_iv: "f10e75e5fff167a439b72456edadc616"
server_key_version: "encryptionkey@0"
I20260806 00:57:58.254871 18438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:57:58.273635 18438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:57:58.276933 18438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:57:58.277169 18438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:57:58.277796 18438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:57:58.278669 18438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:57:58.278743 18438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:58.278816 18438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:57:58.278869 18438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:58.288386 18438 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:42379
I20260806 00:57:58.288411 18605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:42379 every 8 connection(s)
I20260806 00:57:58.289330 18438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:57:58.289978 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 18438
I20260806 00:57:58.290120 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:57:58.290449 27959 external_mini_cluster.cc:1428] Setting key a6defbe3f9524c4584f87af9f1750edb
Aug 06 00:57:58 dist-test-slave-sb2z krb5kdc[18292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977878, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
I20260806 00:57:58.304943 18404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:58.291511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:45723 (local address 127.27.77.254:44551)
0806 00:57:58.291697 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:58.291701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:58.292737 (+ 1036us) server_negotiation.cc:407] Connection header received
0806 00:57:58.294099 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:58.294102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:58.294155 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:58.294236 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:58.295427 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.296057 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.297215 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.297461 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.297587 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:58.300509 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:58.300513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:58.300515 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:57:58.300539 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:58.302350 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:58.302979 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:58.302984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:58.302985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:58.303067 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:58.303468 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:58.303474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:58.303476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:58.303755 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:58.303872 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:58.304592 (+ 720us) server_negotiation.cc:299] Negotiation successful
0806 00:57:58.304744 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 00:57:58.305807 18610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:58.292043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:44551 (local address 127.27.77.193:45723)
0806 00:57:58.292593 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:58.292638 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:58.293888 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:58.294412 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:58.294419 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:58.294783 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:57:58.295277 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:58.295290 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.296193 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.296197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:58.297082 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:58.297093 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.297624 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.297628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:58.297808 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:58.298508 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:57:58.298525 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:57:58.300323 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:57:58.302512 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:58.302518 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:58.302536 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:58.302832 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:58.303177 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:58.303182 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:58.303184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:58.303344 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:58.303885 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:57:58.303894 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:57:58.304292 (+ 398us) client_negotiation.cc:769] Sending connection context
0806 00:57:58.304541 (+ 249us) client_negotiation.cc:240] Negotiation successful
0806 00:57:58.304908 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":154,"threads_started":1}
I20260806 00:57:58.307281 18608 heartbeater.cc:347] Connected to a master server at 127.27.77.254:44551
I20260806 00:57:58.307686 18608 heartbeater.cc:464] Registering TS with master...
I20260806 00:57:58.308370 18608 heartbeater.cc:511] Master 127.27.77.254:44551 requested a full tablet report, sending...
I20260806 00:57:58.310149 18351 ts_manager.cc:194] Registered new tserver with Master: 53bd8ab29cfc491faa405848cbb1bf37 (127.27.77.193:42379)
I20260806 00:57:58.311293 18351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:45723
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:57:58.322816 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:44551
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:57:58.449889 18614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:58.450165 18614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:58.450212 18614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:58.454253 18614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:58.454324 18614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:58.454384 18614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:57:58.460121 18614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:44551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:57:58.461227 18614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:57:58.462230 18614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:57:58.472334 18625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:58.472379 18627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:58.472848 18614 server_base.cc:1034] running on GCE node
W20260806 00:57:58.472406 18620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:58.473630 18614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:57:58.474342 18614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:57:58.475580 18614 hybrid_clock.cc:648] HybridClock initialized: now 1785977878475556 us; error 34 us; skew 500 ppm
Aug 06 00:57:58 dist-test-slave-sb2z krb5kdc[18292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977878, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:57:58.478487 18614 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:57:58.479398 18614 webserver.cc:460] Webserver started at http://127.27.77.194:37413/ using document root <none> and password file <none>
I20260806 00:57:58.480057 18614 fs_manager.cc:359] Metadata directory not provided
I20260806 00:57:58.480176 18614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:57:58.480417 18614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:57:58.482512 18614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "d3f4f9fd4c9b436195cdaf53db053954"
format_stamp: "Formatted at 2026-08-06 00:57:58 on dist-test-slave-sb2z"
server_key: "98ba3aa5d0c34247c86da938253461bb"
server_key_iv: "aa8062ddc896428390687db4e2c86daa"
server_key_version: "encryptionkey@0"
I20260806 00:57:58.483094 18614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "d3f4f9fd4c9b436195cdaf53db053954"
format_stamp: "Formatted at 2026-08-06 00:57:58 on dist-test-slave-sb2z"
server_key: "98ba3aa5d0c34247c86da938253461bb"
server_key_iv: "aa8062ddc896428390687db4e2c86daa"
server_key_version: "encryptionkey@0"
I20260806 00:57:58.487118 18614 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:57:58.489877 18635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:58.491261 18614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:57:58.491411 18614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "d3f4f9fd4c9b436195cdaf53db053954"
format_stamp: "Formatted at 2026-08-06 00:57:58 on dist-test-slave-sb2z"
server_key: "98ba3aa5d0c34247c86da938253461bb"
server_key_iv: "aa8062ddc896428390687db4e2c86daa"
server_key_version: "encryptionkey@0"
I20260806 00:57:58.491569 18614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:57:58.548404 18614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:57:58.552225 18614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:57:58.552484 18614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:57:58.553192 18614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:57:58.554203 18614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:57:58.554303 18614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:58.554405 18614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:57:58.554464 18614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:58.565399 18614 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:43839
I20260806 00:57:58.565428 18778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:43839 every 8 connection(s)
I20260806 00:57:58.566462 18614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:57:58.571861 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 18614
I20260806 00:57:58.572073 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:57:58.572413 27959 external_mini_cluster.cc:1428] Setting key b290108ffae9686de24783120f1e4b91
Aug 06 00:57:58 dist-test-slave-sb2z krb5kdc[18292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977878, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:57:58.583608 18404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:58.568677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:41323 (local address 127.27.77.254:44551)
0806 00:57:58.568870 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:58.568875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:58.570000 (+ 1125us) server_negotiation.cc:407] Connection header received
0806 00:57:58.571394 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:58.571397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:58.571454 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:58.571653 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:58.573163 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.574004 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.575854 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.576102 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.576193 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:58.579492 (+ 3299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:58.579497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:58.579502 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:57:58.579586 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:58.581268 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:58.581880 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:58.581884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:58.581886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:58.581987 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:58.582359 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:58.582363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:58.582365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:58.582577 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:58.582689 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:58.583264 (+ 575us) server_negotiation.cc:299] Negotiation successful
0806 00:57:58.583407 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
I20260806 00:57:58.584568 18781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:58.569094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:44551 (local address 127.27.77.194:41323)
0806 00:57:58.569824 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:58.569870 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:58.571159 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:58.571833 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:58.571840 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:58.572366 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:57:58.572935 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:58.572951 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.574193 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.574197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:58.575701 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:58.575717 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.576278 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.576285 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:58.576546 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:58.577300 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:57:58.577318 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:57:58.579296 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:57:58.581431 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:58.581437 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:58.581467 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:58.581758 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:58.582089 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:58.582093 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:58.582095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:58.582253 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:58.582700 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:57:58.582709 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:57:58.583007 (+ 298us) client_negotiation.cc:769] Sending connection context
0806 00:57:58.583219 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 00:57:58.583618 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":134,"threads_started":1}
I20260806 00:57:58.586054 18779 heartbeater.cc:347] Connected to a master server at 127.27.77.254:44551
I20260806 00:57:58.586380 18779 heartbeater.cc:464] Registering TS with master...
I20260806 00:57:58.587118 18779 heartbeater.cc:511] Master 127.27.77.254:44551 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
I20260806 00:57:58.588551 18351 ts_manager.cc:194] Registered new tserver with Master: d3f4f9fd4c9b436195cdaf53db053954 (127.27.77.194:43839)
I20260806 00:57:58.589267 18351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:41323
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:57:58.607888 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:44551
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:57:58.727415 18784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:58.727761 18784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:58.727811 18784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:58.731961 18784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:58.732024 18784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:58.732091 18784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:57:58.737488 18784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:44551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:57:58.738700 18784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:57:58.739854 18784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:57:58.750541 18791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:58.750502 18792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:58.750687 18784 server_base.cc:1034] running on GCE node
W20260806 00:57:58.750598 18795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:58.751343 18784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:57:58.752113 18784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:57:58.753306 18784 hybrid_clock.cc:648] HybridClock initialized: now 1785977878753278 us; error 41 us; skew 500 ppm
Aug 06 00:57:58 dist-test-slave-sb2z krb5kdc[18292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977878, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:57:58.756729 18784 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:57:58.757807 18784 webserver.cc:460] Webserver started at http://127.27.77.195:34283/ using document root <none> and password file <none>
I20260806 00:57:58.758463 18784 fs_manager.cc:359] Metadata directory not provided
I20260806 00:57:58.758582 18784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:57:58.758906 18784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:57:58.761365 18784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "26d8148a58a94af6806e381023c3736d"
format_stamp: "Formatted at 2026-08-06 00:57:58 on dist-test-slave-sb2z"
server_key: "25696a9d94f6fbb1de626a3b89fba92a"
server_key_iv: "e342643cb44072cd2ca4be9f63efb804"
server_key_version: "encryptionkey@0"
I20260806 00:57:58.762033 18784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "26d8148a58a94af6806e381023c3736d"
format_stamp: "Formatted at 2026-08-06 00:57:58 on dist-test-slave-sb2z"
server_key: "25696a9d94f6fbb1de626a3b89fba92a"
server_key_iv: "e342643cb44072cd2ca4be9f63efb804"
server_key_version: "encryptionkey@0"
I20260806 00:57:58.766340 18784 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260806 00:57:58.769215 18809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:58.770569 18784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 00:57:58.770787 18784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "26d8148a58a94af6806e381023c3736d"
format_stamp: "Formatted at 2026-08-06 00:57:58 on dist-test-slave-sb2z"
server_key: "25696a9d94f6fbb1de626a3b89fba92a"
server_key_iv: "e342643cb44072cd2ca4be9f63efb804"
server_key_version: "encryptionkey@0"
I20260806 00:57:58.770921 18784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:57:58.819228 18784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:57:58.822862 18784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:57:58.823102 18784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:57:58.824157 18784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:57:58.825198 18784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:57:58.825289 18784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:58.825373 18784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:57:58.825417 18784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:58.835599 18784 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:43195
I20260806 00:57:58.835637 18961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:43195 every 8 connection(s)
I20260806 00:57:58.836692 18784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
Aug 06 00:57:58 dist-test-slave-sb2z krb5kdc[18292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977878, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:57:58.846992 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 18784
I20260806 00:57:58.847132 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:57:58.847492 27959 external_mini_cluster.cc:1428] Setting key 0f4340b7bedcd19bf4484011a3d18300
I20260806 00:57:58.852072 18404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:58.838699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:39321 (local address 127.27.77.254:44551)
0806 00:57:58.838867 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:58.838872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:58.840354 (+ 1482us) server_negotiation.cc:407] Connection header received
0806 00:57:58.841266 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:58.841269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:58.841320 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:58.841403 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:58.842644 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.843192 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.844485 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.844725 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.844822 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:58.847772 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:58.847775 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:58.847777 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:57:58.847805 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:58.849706 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:58.850414 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:58.850418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:58.850419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:58.850507 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:58.850854 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:58.850856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:58.850857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:58.851114 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:58.851195 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:58.851801 (+ 606us) server_negotiation.cc:299] Negotiation successful
0806 00:57:58.851911 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 00:57:58.852849 18965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:58.839164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:44551 (local address 127.27.77.195:39321)
0806 00:57:58.840187 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:58.840232 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:58.841055 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:58.841597 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:58.841603 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:58.841963 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:57:58.842464 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:58.842477 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.843328 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.843331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:58.844351 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:58.844362 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.844877 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.844879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:58.845024 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:58.845690 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:57:58.845707 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:57:58.847513 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:57:58.849838 (+ 2325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:58.849844 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:58.849857 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:58.850281 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:58.850607 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:57:58.850609 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:57:58.850610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:57:58.850752 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:57:58.851209 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:57:58.851214 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:57:58.851463 (+ 249us) client_negotiation.cc:769] Sending connection context
0806 00:57:58.851686 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 00:57:58.851969 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"thread_start_us":652,"threads_started":1}
I20260806 00:57:58.854084 18963 heartbeater.cc:347] Connected to a master server at 127.27.77.254:44551
I20260806 00:57:58.854370 18963 heartbeater.cc:464] Registering TS with master...
I20260806 00:57:58.854928 18963 heartbeater.cc:511] Master 127.27.77.254:44551 requested a full tablet report, sending...
I20260806 00:57:58.856148 18351 ts_manager.cc:194] Registered new tserver with Master: 26d8148a58a94af6806e381023c3736d (127.27.77.195:43195)
I20260806 00:57:58.856802 18351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:39321
I20260806 00:57:58.862268 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:57:58.872388 18404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:58.864332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42772 (local address 127.27.77.254:44551)
0806 00:57:58.864500 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:58.864505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:58.864719 (+ 214us) server_negotiation.cc:407] Connection header received
0806 00:57:58.864770 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:58.864772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:58.864817 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:58.864899 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:58.865634 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.866224 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.867102 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.867339 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.867430 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:58.869045 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:58.869048 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:58.869050 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:57:58.869075 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:58.870580 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:58.871008 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:58.871011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:58.871013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:58.871103 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:57:58.871356 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:57:58.871359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:57:58.871360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:57:58.871716 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:58.871834 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:58.872044 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 00:57:58.872149 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 00:57:58.875468 18351 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:57:58.878535 18351 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:57:58.893486 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:58.887262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:42379 (local address 127.0.0.1:44000)
0806 00:57:58.888466 (+ 1204us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:58.888484 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:58.888586 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:58.889316 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:58.889319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:58.889335 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:57:58.889475 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:58.889482 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.890490 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.890497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:58.891818 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:58.891828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.892906 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.892909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:58.893029 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:58.893081 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 00:57:58.893197 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 00:57:58.893322 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1104,"mutex_wait_us":25,"thread_start_us":130,"threads_started":1}
I20260806 00:57:58.893603 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:58.887931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44000 (local address 127.27.77.193:42379)
0806 00:57:58.888395 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:58.888403 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:58.888568 (+ 165us) server_negotiation.cc:407] Connection header received
0806 00:57:58.888738 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:58.888744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:58.888972 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:58.889150 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:57:58.889624 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.890364 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.891942 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.892539 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.892673 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:58.892835 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:58.893276 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 00:57:58.893398 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":134,"threads_started":1}
I20260806 00:57:58.894579 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:58.887049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:43195 (local address 127.0.0.1:45306)
0806 00:57:58.887907 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:58.887963 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:58.888164 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:58.889157 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:58.889160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:58.889195 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:57:58.889372 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:58.889378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.891025 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.891029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:58.892500 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:58.892513 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.893542 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.893546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:58.894240 (+ 694us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:58.894266 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:57:58.894342 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 00:57:58.894421 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":55,"spinlock_wait_cycles":3200,"thread_start_us":93,"threads_started":1}
I20260806 00:57:58.894784 18981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:58.887275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45306 (local address 127.27.77.195:43195)
0806 00:57:58.888634 (+ 1359us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:58.888649 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:58.888673 (+ 24us) server_negotiation.cc:407] Connection header received
0806 00:57:58.888744 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:58.888748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:58.888972 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:58.889266 (+ 294us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:57:58.889470 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.890823 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.892681 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.893388 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.894197 (+ 809us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:58.894421 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:58.894484 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 00:57:58.894602 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1219,"thread_start_us":155,"threads_started":1}
I20260806 00:57:58.895357 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:58.887441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:43839 (local address 127.0.0.1:45184)
0806 00:57:58.888586 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:58.888598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:58.888681 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:58.889150 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:58.889157 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:58.889193 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:57:58.889520 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:58.889524 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.891089 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.891092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:58.892511 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:58.892519 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.893862 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.893866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:58.894928 (+ 1062us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:58.895019 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 00:57:58.895113 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 00:57:58.895191 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1091,"mutex_wait_us":350,"spinlock_wait_cycles":6272,"thread_start_us":126,"threads_started":1}
I20260806 00:57:58.895853 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:58.887927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45184 (local address 127.27.77.194:43839)
0806 00:57:58.888493 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:58.888505 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:58.888772 (+ 267us) server_negotiation.cc:407] Connection header received
0806 00:57:58.888844 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:58.888847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:58.888985 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:58.889156 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:57:58.889652 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.890923 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.892650 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:58.893496 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:58.893631 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:58.893796 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:58.895582 (+ 1786us) server_negotiation.cc:299] Negotiation successful
0806 00:57:58.895701 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":177,"threads_started":1}
I20260806 00:57:58.897239 18518 tablet_service.cc:1467] Processing CreateTablet for tablet 2fcd02ab46304e91ab022ed8ae91d1d5 (DEFAULT_TABLE table=test-table [id=d276597994e543918ad549316b4d4a08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:57:58.897564 18875 tablet_service.cc:1467] Processing CreateTablet for tablet 2fcd02ab46304e91ab022ed8ae91d1d5 (DEFAULT_TABLE table=test-table [id=d276597994e543918ad549316b4d4a08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:57:58.898437 18518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fcd02ab46304e91ab022ed8ae91d1d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:57:58.898489 18875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fcd02ab46304e91ab022ed8ae91d1d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:57:58.898505 18694 tablet_service.cc:1467] Processing CreateTablet for tablet 2fcd02ab46304e91ab022ed8ae91d1d5 (DEFAULT_TABLE table=test-table [id=d276597994e543918ad549316b4d4a08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:57:58.899459 18694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fcd02ab46304e91ab022ed8ae91d1d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:57:58.905081 18986 tablet_bootstrap.cc:492] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d: Bootstrap starting.
I20260806 00:57:58.906580 18988 tablet_bootstrap.cc:492] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37: Bootstrap starting.
I20260806 00:57:58.906957 18989 tablet_bootstrap.cc:492] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954: Bootstrap starting.
I20260806 00:57:58.908087 18986 tablet_bootstrap.cc:654] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d: Neither blocks nor log segments found. Creating new log.
I20260806 00:57:58.908831 18988 tablet_bootstrap.cc:654] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37: Neither blocks nor log segments found. Creating new log.
I20260806 00:57:58.909598 18989 tablet_bootstrap.cc:654] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954: Neither blocks nor log segments found. Creating new log.
I20260806 00:57:58.909664 18986 log.cc:824] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d: Log is configured to *not* fsync() on all Append() calls
I20260806 00:57:58.909737 18988 log.cc:824] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37: Log is configured to *not* fsync() on all Append() calls
I20260806 00:57:58.911468 18989 log.cc:824] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954: Log is configured to *not* fsync() on all Append() calls
I20260806 00:57:58.911984 18986 tablet_bootstrap.cc:492] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d: No bootstrap required, opened a new log
I20260806 00:57:58.911984 18988 tablet_bootstrap.cc:492] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37: No bootstrap required, opened a new log
I20260806 00:57:58.912195 18988 ts_tablet_manager.cc:1397] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:57:58.912195 18986 ts_tablet_manager.cc:1397] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 00:57:58.913400 18989 tablet_bootstrap.cc:492] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954: No bootstrap required, opened a new log
I20260806 00:57:58.913630 18989 ts_tablet_manager.cc:1397] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:57:58.916874 18988 raft_consensus.cc:348] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26d8148a58a94af6806e381023c3736d" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43195 } } peers { permanent_uuid: "53bd8ab29cfc491faa405848cbb1bf37" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42379 } } peers { permanent_uuid: "d3f4f9fd4c9b436195cdaf53db053954" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43839 } }
I20260806 00:57:58.916901 18989 raft_consensus.cc:348] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26d8148a58a94af6806e381023c3736d" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43195 } } peers { permanent_uuid: "53bd8ab29cfc491faa405848cbb1bf37" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42379 } } peers { permanent_uuid: "d3f4f9fd4c9b436195cdaf53db053954" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43839 } }
I20260806 00:57:58.917171 18988 raft_consensus.cc:374] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:57:58.917225 18989 raft_consensus.cc:374] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:57:58.917234 18988 raft_consensus.cc:729] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53bd8ab29cfc491faa405848cbb1bf37, State: Initialized, Role: FOLLOWER
I20260806 00:57:58.917279 18989 raft_consensus.cc:729] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3f4f9fd4c9b436195cdaf53db053954, State: Initialized, Role: FOLLOWER
I20260806 00:57:58.917397 18986 raft_consensus.cc:348] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26d8148a58a94af6806e381023c3736d" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43195 } } peers { permanent_uuid: "53bd8ab29cfc491faa405848cbb1bf37" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42379 } } peers { permanent_uuid: "d3f4f9fd4c9b436195cdaf53db053954" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43839 } }
I20260806 00:57:58.917632 18986 raft_consensus.cc:374] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:57:58.917686 18986 raft_consensus.cc:729] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26d8148a58a94af6806e381023c3736d, State: Initialized, Role: FOLLOWER
I20260806 00:57:58.917724 18988 consensus_queue.cc:260] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26d8148a58a94af6806e381023c3736d" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43195 } } peers { permanent_uuid: "53bd8ab29cfc491faa405848cbb1bf37" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42379 } } peers { permanent_uuid: "d3f4f9fd4c9b436195cdaf53db053954" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43839 } }
I20260806 00:57:58.917830 18989 consensus_queue.cc:260] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26d8148a58a94af6806e381023c3736d" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43195 } } peers { permanent_uuid: "53bd8ab29cfc491faa405848cbb1bf37" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42379 } } peers { permanent_uuid: "d3f4f9fd4c9b436195cdaf53db053954" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43839 } }
I20260806 00:57:58.918056 18986 consensus_queue.cc:260] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26d8148a58a94af6806e381023c3736d" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43195 } } peers { permanent_uuid: "53bd8ab29cfc491faa405848cbb1bf37" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42379 } } peers { permanent_uuid: "d3f4f9fd4c9b436195cdaf53db053954" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43839 } }
I20260806 00:57:58.918540 18608 heartbeater.cc:503] Master 127.27.77.254:44551 was elected leader, sending a full tablet report...
I20260806 00:57:58.918815 18988 ts_tablet_manager.cc:1428] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 00:57:58.918887 18986 ts_tablet_manager.cc:1428] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 00:57:58.918900 18989 ts_tablet_manager.cc:1428] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 00:57:58.919206 18963 heartbeater.cc:503] Master 127.27.77.254:44551 was elected leader, sending a full tablet report...
I20260806 00:57:58.919790 18779 heartbeater.cc:503] Master 127.27.77.254:44551 was elected leader, sending a full tablet report...
W20260806 00:57:59.041003 18609 tablet.cc:2369] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:57:59.051904 18994 raft_consensus.cc:482] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:57:59.052127 18994 raft_consensus.cc:504] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26d8148a58a94af6806e381023c3736d" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43195 } } peers { permanent_uuid: "53bd8ab29cfc491faa405848cbb1bf37" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42379 } } peers { permanent_uuid: "d3f4f9fd4c9b436195cdaf53db053954" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43839 } }
I20260806 00:57:59.053440 18994 leader_election.cc:288] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26d8148a58a94af6806e381023c3736d (127.27.77.195:43195), d3f4f9fd4c9b436195cdaf53db053954 (127.27.77.194:43839)
I20260806 00:57:59.057890 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:59.054056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:47189 (local address 127.27.77.194:43839)
0806 00:57:59.054200 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:59.054205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:59.054383 (+ 178us) server_negotiation.cc:407] Connection header received
0806 00:57:59.054528 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:59.054531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:59.054586 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:59.054673 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:57:59.055120 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.055693 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.056835 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.057271 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.057368 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:59.057410 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:59.057660 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 00:57:59.057740 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 00:57:59.057941 18997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:59.053793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:43839 (local address 127.27.77.193:47189)
0806 00:57:59.054266 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:59.054280 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:59.054418 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:59.054717 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:59.054721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:59.054736 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:57:59.054934 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:59.054944 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.055800 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.055803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:59.056694 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:59.056705 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.057409 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.057413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:59.057521 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:59.057552 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 00:57:59.057626 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 00:57:59.057721 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":3,"thread_start_us":131,"threads_started":1}
I20260806 00:57:59.058080 18981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:59.054004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:35773 (local address 127.27.77.195:43195)
0806 00:57:59.054157 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:59.054162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:59.054558 (+ 396us) server_negotiation.cc:407] Connection header received
0806 00:57:59.054697 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:59.054700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:59.054760 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:59.054847 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:57:59.055255 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.055873 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.056956 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.057443 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.057564 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:59.057634 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:59.057861 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 00:57:59.057947 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:57:59.058466 18998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:59.053793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:43195 (local address 127.27.77.193:35773)
0806 00:57:59.054408 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:59.054424 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:59.054559 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:59.054882 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:59.054884 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:59.054900 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:57:59.055125 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:59.055134 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.055996 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.055999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:59.056827 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:59.056837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.057584 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.057587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:59.057704 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:59.057727 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:57:59.057803 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 00:57:59.057868 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":199,"thread_start_us":97,"threads_started":1}
I20260806 00:57:59.058727 18722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcd02ab46304e91ab022ed8ae91d1d5" candidate_uuid: "53bd8ab29cfc491faa405848cbb1bf37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3f4f9fd4c9b436195cdaf53db053954" is_pre_election: true
I20260806 00:57:59.059080 18722 raft_consensus.cc:2454] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53bd8ab29cfc491faa405848cbb1bf37 in term 0.
I20260806 00:57:59.059137 18903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcd02ab46304e91ab022ed8ae91d1d5" candidate_uuid: "53bd8ab29cfc491faa405848cbb1bf37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26d8148a58a94af6806e381023c3736d" is_pre_election: true
I20260806 00:57:59.059464 18903 raft_consensus.cc:2454] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53bd8ab29cfc491faa405848cbb1bf37 in term 0.
I20260806 00:57:59.059746 18459 leader_election.cc:302] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53bd8ab29cfc491faa405848cbb1bf37, d3f4f9fd4c9b436195cdaf53db053954; no voters:
I20260806 00:57:59.060051 18994 raft_consensus.cc:2781] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:57:59.060148 18994 raft_consensus.cc:482] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:57:59.060196 18994 raft_consensus.cc:3037] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:57:59.061379 18994 raft_consensus.cc:504] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26d8148a58a94af6806e381023c3736d" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43195 } } peers { permanent_uuid: "53bd8ab29cfc491faa405848cbb1bf37" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42379 } } peers { permanent_uuid: "d3f4f9fd4c9b436195cdaf53db053954" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43839 } }
I20260806 00:57:59.061748 18994 leader_election.cc:288] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [CANDIDATE]: Term 1 election: Requested vote from peers 26d8148a58a94af6806e381023c3736d (127.27.77.195:43195), d3f4f9fd4c9b436195cdaf53db053954 (127.27.77.194:43839)
I20260806 00:57:59.062143 18903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcd02ab46304e91ab022ed8ae91d1d5" candidate_uuid: "53bd8ab29cfc491faa405848cbb1bf37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26d8148a58a94af6806e381023c3736d"
I20260806 00:57:59.062307 18903 raft_consensus.cc:3037] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:57:59.062443 18722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcd02ab46304e91ab022ed8ae91d1d5" candidate_uuid: "53bd8ab29cfc491faa405848cbb1bf37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3f4f9fd4c9b436195cdaf53db053954"
I20260806 00:57:59.062618 18722 raft_consensus.cc:3037] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:57:59.063818 18903 raft_consensus.cc:2454] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53bd8ab29cfc491faa405848cbb1bf37 in term 1.
I20260806 00:57:59.064137 18722 raft_consensus.cc:2454] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53bd8ab29cfc491faa405848cbb1bf37 in term 1.
I20260806 00:57:59.064189 18456 leader_election.cc:302] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26d8148a58a94af6806e381023c3736d, 53bd8ab29cfc491faa405848cbb1bf37; no voters:
I20260806 00:57:59.064380 18994 raft_consensus.cc:2781] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:57:59.064683 18994 raft_consensus.cc:686] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [term 1 LEADER]: Becoming Leader. State: Replica: 53bd8ab29cfc491faa405848cbb1bf37, State: Running, Role: LEADER
I20260806 00:57:59.065049 18994 consensus_queue.cc:237] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26d8148a58a94af6806e381023c3736d" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43195 } } peers { permanent_uuid: "53bd8ab29cfc491faa405848cbb1bf37" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42379 } } peers { permanent_uuid: "d3f4f9fd4c9b436195cdaf53db053954" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43839 } }
W20260806 00:57:59.067756 18780 tablet.cc:2369] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:57:59.068688 18350 catalog_manager.cc:5495] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53bd8ab29cfc491faa405848cbb1bf37 (127.27.77.193). New cstate: current_term: 1 leader_uuid: "53bd8ab29cfc491faa405848cbb1bf37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26d8148a58a94af6806e381023c3736d" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53bd8ab29cfc491faa405848cbb1bf37" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3f4f9fd4c9b436195cdaf53db053954" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43839 } health_report { overall_health: UNKNOWN } } }
W20260806 00:57:59.087922 18964 tablet.cc:2369] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:57:59.091166 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:59.087369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44004 (local address 127.27.77.193:42379)
0806 00:57:59.087512 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:59.087516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:59.087571 (+ 55us) server_negotiation.cc:407] Connection header received
0806 00:57:59.087687 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:59.087689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:59.087749 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:59.087837 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:57:59.088225 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.088744 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.089636 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.089853 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.089936 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:59.090300 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:57:59.090726 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:57:59.090831 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:59.090895 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 00:57:59.090954 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:57:59.097827 18722 raft_consensus.cc:1261] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954 [term 1 FOLLOWER]: Refusing update from remote peer 53bd8ab29cfc491faa405848cbb1bf37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:57:59.097828 18903 raft_consensus.cc:1261] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d [term 1 FOLLOWER]: Refusing update from remote peer 53bd8ab29cfc491faa405848cbb1bf37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:57:59.098652 18999 consensus_queue.cc:1035] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3f4f9fd4c9b436195cdaf53db053954" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:57:59.098946 18994 consensus_queue.cc:1035] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 53bd8ab29cfc491faa405848cbb1bf37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26d8148a58a94af6806e381023c3736d" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 43195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:57:59.118896 18351 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:57:59.119114 18351 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:57:59.121847 18351 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c3f054073d464242964a1c6bc62a21be: Sending DeleteTablet for 3 replicas of tablet 2fcd02ab46304e91ab022ed8ae91d1d5
I20260806 00:57:59.122751 18518 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fcd02ab46304e91ab022ed8ae91d1d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:57:59 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:44000
I20260806 00:57:59.122759 18875 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fcd02ab46304e91ab022ed8ae91d1d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:57:59 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:45306
I20260806 00:57:59.122759 18694 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fcd02ab46304e91ab022ed8ae91d1d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:57:59 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:45184
I20260806 00:57:59.123517 19018 tablet_replica.cc:330] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954: stopping tablet replica
I20260806 00:57:59.123842 19016 tablet_replica.cc:330] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d: stopping tablet replica
I20260806 00:57:59.123915 19018 raft_consensus.cc:2229] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:57:59.124291 19018 raft_consensus.cc:2258] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:57:59.124336 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 18438
I20260806 00:57:59.124297 19016 raft_consensus.cc:2229] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:57:59.125473 19016 raft_consensus.cc:2258] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:57:59.128123 19018 ts_tablet_manager.cc:1905] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:57:59.131698 19018 ts_tablet_manager.cc:1918] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:57:59.131820 19018 log.cc:1196] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/2fcd02ab46304e91ab022ed8ae91d1d5
I20260806 00:57:59.132196 19018 ts_tablet_manager.cc:1939] T 2fcd02ab46304e91ab022ed8ae91d1d5 P d3f4f9fd4c9b436195cdaf53db053954: Deleting consensus metadata
I20260806 00:57:59.133029 19016 ts_tablet_manager.cc:1905] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:57:59.133708 18335 catalog_manager.cc:4841] TS d3f4f9fd4c9b436195cdaf53db053954 (127.27.77.194:43839): tablet 2fcd02ab46304e91ab022ed8ae91d1d5 (table test-table [id=d276597994e543918ad549316b4d4a08]) successfully deleted
I20260806 00:57:59.135740 19016 ts_tablet_manager.cc:1918] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:57:59.135874 19016 log.cc:1196] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/2fcd02ab46304e91ab022ed8ae91d1d5
I20260806 00:57:59.136235 19016 ts_tablet_manager.cc:1939] T 2fcd02ab46304e91ab022ed8ae91d1d5 P 26d8148a58a94af6806e381023c3736d: Deleting consensus metadata
W20260806 00:57:59.136966 18334 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:42379 (error 108)
I20260806 00:57:59.137650 18333 catalog_manager.cc:4841] TS 26d8148a58a94af6806e381023c3736d (127.27.77.195:43195): tablet 2fcd02ab46304e91ab022ed8ae91d1d5 (table test-table [id=d276597994e543918ad549316b4d4a08]) successfully deleted
I20260806 00:57:59.137935 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 18614
I20260806 00:57:59.138147 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:59.137779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:42379 (local address 127.0.0.1:44014)
0806 00:57:59.137955 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:59.138049 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:42379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260806 00:57:59.140846 18334 catalog_manager.cc:4584] TS 53bd8ab29cfc491faa405848cbb1bf37 (127.27.77.193:42379): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2fcd02ab46304e91ab022ed8ae91d1d5: Network error: Client connection negotiation failed: client connection to 127.27.77.193:42379: connect: Connection refused (error 111)
I20260806 00:57:59.151463 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 18784
I20260806 00:57:59.162650 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 18312
2026-08-06T00:57:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2777 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T00:57:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:57:59Z Disabled control of system clock
I20260806 00:57:59.202940 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:46515
--webserver_interface=127.27.77.254
--webserver_port=0
--builtin_ntp_servers=127.27.77.212:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:46515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:57:59.315809 19023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:59.316147 19023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:59.316187 19023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:59.323340 19023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:57:59.323448 19023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:59.323478 19023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:59.323541 19023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:57:59.323571 19023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:57:59.328704 19023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:46515
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:46515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:57:59.329878 19023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:57:59.331293 19023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:57:59.339834 19031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:59.339846 19028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:59.339900 19023 server_base.cc:1034] running on GCE node
W20260806 00:57:59.339845 19033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:59.340643 19023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:57:59.341795 19023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:57:59.342998 19023 hybrid_clock.cc:648] HybridClock initialized: now 1785977879342971 us; error 40 us; skew 500 ppm
I20260806 00:57:59.345041 19023 webserver.cc:460] Webserver started at http://127.27.77.254:32779/ using document root <none> and password file <none>
I20260806 00:57:59.345669 19023 fs_manager.cc:359] Metadata directory not provided
I20260806 00:57:59.345769 19023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:57:59.346021 19023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:57:59.348234 19023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "4a7478eee528437ca4e237fee55e94c8"
format_stamp: "Formatted at 2026-08-06 00:57:59 on dist-test-slave-sb2z"
server_key: "332996a7a4936bdec5542a181728e527"
server_key_iv: "b687ca5f9010f00fb2621137c0bb891c"
server_key_version: "encryptionkey@0"
I20260806 00:57:59.348831 19023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "4a7478eee528437ca4e237fee55e94c8"
format_stamp: "Formatted at 2026-08-06 00:57:59 on dist-test-slave-sb2z"
server_key: "332996a7a4936bdec5542a181728e527"
server_key_iv: "b687ca5f9010f00fb2621137c0bb891c"
server_key_version: "encryptionkey@0"
I20260806 00:57:59.352644 19023 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:57:59.355129 19043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:59.356945 19023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:57:59.357124 19023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "4a7478eee528437ca4e237fee55e94c8"
format_stamp: "Formatted at 2026-08-06 00:57:59 on dist-test-slave-sb2z"
server_key: "332996a7a4936bdec5542a181728e527"
server_key_iv: "b687ca5f9010f00fb2621137c0bb891c"
server_key_version: "encryptionkey@0"
I20260806 00:57:59.357276 19023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:57:59.369602 19023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:57:59.370337 19023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:57:59.370556 19023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:57:59.377956 19023 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:46515
I20260806 00:57:59.377966 19118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:46515 every 8 connection(s)
I20260806 00:57:59.378978 19023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:57:59.379746 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 19023
I20260806 00:57:59.379894 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:57:59.380198 27959 external_mini_cluster.cc:1428] Setting key 1903bc8d8eb941f4ef7e00323d02cf0d
I20260806 00:57:59.383173 19119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:57:59.391156 19119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8: Bootstrap starting.
I20260806 00:57:59.392086 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:59.381756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54836 (local address 127.27.77.254:46515)
0806 00:57:59.382481 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:59.382505 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:59.382550 (+ 45us) server_negotiation.cc:407] Connection header received
0806 00:57:59.383563 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:59.383590 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:59.384147 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:59.384537 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:59.385027 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.386679 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.387728 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.388490 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.388690 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:59.389673 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:59.389681 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:59.389700 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:57:59.390294 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:59.390370 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:57:59.390398 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:59.390512 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:59.390694 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 00:57:59.391019 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":125,"threads_started":1}
I20260806 00:57:59.393935 19119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8: Neither blocks nor log segments found. Creating new log.
I20260806 00:57:59.394838 19119 log.cc:824] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8: Log is configured to *not* fsync() on all Append() calls
I20260806 00:57:59.396875 19119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8: No bootstrap required, opened a new log
I20260806 00:57:59.400601 19119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7478eee528437ca4e237fee55e94c8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46515 } }
I20260806 00:57:59.400822 19119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:57:59.400859 19119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a7478eee528437ca4e237fee55e94c8, State: Initialized, Role: FOLLOWER
I20260806 00:57:59.401234 19119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7478eee528437ca4e237fee55e94c8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46515 } }
I20260806 00:57:59.401347 19119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:57:59.401392 19119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:57:59.401471 19119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:57:59.402352 19119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7478eee528437ca4e237fee55e94c8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46515 } }
I20260806 00:57:59.402660 19119 leader_election.cc:302] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a7478eee528437ca4e237fee55e94c8; no voters:
I20260806 00:57:59.402920 19119 leader_election.cc:288] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:57:59.403061 19127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:57:59.403337 19127 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [term 1 LEADER]: Becoming Leader. State: Replica: 4a7478eee528437ca4e237fee55e94c8, State: Running, Role: LEADER
I20260806 00:57:59.403690 19127 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7478eee528437ca4e237fee55e94c8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46515 } }
I20260806 00:57:59.404206 19119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:57:59.405279 19128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a7478eee528437ca4e237fee55e94c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7478eee528437ca4e237fee55e94c8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46515 } } }
I20260806 00:57:59.405416 19128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [sys.catalog]: This master's current role is: LEADER
I20260806 00:57:59.405776 19129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a7478eee528437ca4e237fee55e94c8. Latest consensus state: current_term: 1 leader_uuid: "4a7478eee528437ca4e237fee55e94c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7478eee528437ca4e237fee55e94c8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46515 } } }
I20260806 00:57:59.405884 19129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8 [sys.catalog]: This master's current role is: LEADER
I20260806 00:57:59.406134 19136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:57:59.409103 19136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:57:59.414170 19136 catalog_manager.cc:1347] Generated new cluster ID: 085057fb277143f08102a4a68903c0e2
I20260806 00:57:59.414238 19136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:57:59.422788 19136 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:57:59.423864 19136 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:57:59.433372 19136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8: Generated new TSK 0
I20260806 00:57:59.434101 19136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:57:59.443754 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:46515
--builtin_ntp_servers=127.27.77.212:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:57:59.575559 19149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:59.575837 19149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:59.575877 19149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:59.580050 19149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:59.580116 19149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:59.580171 19149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:57:59.585134 19149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:46515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:57:59.586329 19149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:57:59.587899 19149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:57:59.598690 19163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:59.598740 19160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:59.598887 19149 server_base.cc:1034] running on GCE node
W20260806 00:57:59.598694 19159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:59.599700 19149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:57:59.600397 19149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:57:59.601593 19149 hybrid_clock.cc:648] HybridClock initialized: now 1785977879601551 us; error 54 us; skew 500 ppm
I20260806 00:57:59.603897 19149 webserver.cc:460] Webserver started at http://127.27.77.193:41751/ using document root <none> and password file <none>
I20260806 00:57:59.604540 19149 fs_manager.cc:359] Metadata directory not provided
I20260806 00:57:59.604652 19149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:57:59.604900 19149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:57:59.607156 19149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "349154c3a8564d549fc4bac1fd3ea426"
format_stamp: "Formatted at 2026-08-06 00:57:59 on dist-test-slave-sb2z"
server_key: "c09023844e9356cd7f7e5f63e7cd83c5"
server_key_iv: "207b5b74f7541a449f7d20b2624a4f65"
server_key_version: "encryptionkey@0"
I20260806 00:57:59.607806 19149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "349154c3a8564d549fc4bac1fd3ea426"
format_stamp: "Formatted at 2026-08-06 00:57:59 on dist-test-slave-sb2z"
server_key: "c09023844e9356cd7f7e5f63e7cd83c5"
server_key_iv: "207b5b74f7541a449f7d20b2624a4f65"
server_key_version: "encryptionkey@0"
I20260806 00:57:59.611781 19149 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:57:59.614485 19177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:59.615926 19149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:57:59.616081 19149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "349154c3a8564d549fc4bac1fd3ea426"
format_stamp: "Formatted at 2026-08-06 00:57:59 on dist-test-slave-sb2z"
server_key: "c09023844e9356cd7f7e5f63e7cd83c5"
server_key_iv: "207b5b74f7541a449f7d20b2624a4f65"
server_key_version: "encryptionkey@0"
I20260806 00:57:59.616217 19149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:57:59.650978 19149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:57:59.652020 19149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:57:59.652252 19149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:57:59.653086 19149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:57:59.654158 19149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:57:59.654237 19149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:59.654309 19149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:57:59.654340 19149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:59.664499 19149 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:36007
I20260806 00:57:59.664558 19338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:36007 every 8 connection(s)
I20260806 00:57:59.665504 19149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:57:59.672248 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 19149
I20260806 00:57:59.672387 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:57:59.672725 27959 external_mini_cluster.cc:1428] Setting key eaba09ae64b97ce755547549cde7a9ef
I20260806 00:57:59.675130 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:46515
--builtin_ntp_servers=127.27.77.212:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:57:59.676934 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:59.667710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:50727 (local address 127.27.77.254:46515)
0806 00:57:59.667869 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:59.667875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:59.669024 (+ 1149us) server_negotiation.cc:407] Connection header received
0806 00:57:59.670575 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:59.670578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:59.670634 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:59.670717 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:59.671860 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.672739 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.674485 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.674800 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.674935 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:59.676275 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:59.676282 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:59.676289 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:57:59.676308 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:59.676336 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:57:59.676351 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:59.676434 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:59.676607 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 00:57:59.676711 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:57:59.678049 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:59.668137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:46515 (local address 127.27.77.193:50727)
0806 00:57:59.668852 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:59.668907 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:59.670343 (+ 1436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:59.670902 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:59.670909 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:59.671008 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:57:59.671686 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:59.671705 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.672909 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.672912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:59.674328 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:59.674342 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.674974 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.674977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:59.675147 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:59.675975 (+ 828us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:57:59.675977 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:57:59.676023 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:57:59.676103 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:57:59.676445 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:57:59.676454 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:57:59.676459 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 00:57:59.676560 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 00:57:59.676841 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":125,"threads_started":1}
I20260806 00:57:59.679636 19339 heartbeater.cc:347] Connected to a master server at 127.27.77.254:46515
I20260806 00:57:59.679932 19339 heartbeater.cc:464] Registering TS with master...
I20260806 00:57:59.680564 19339 heartbeater.cc:511] Master 127.27.77.254:46515 requested a full tablet report, sending...
I20260806 00:57:59.682385 19072 ts_manager.cc:194] Registered new tserver with Master: 349154c3a8564d549fc4bac1fd3ea426 (127.27.77.193:36007)
I20260806 00:57:59.683745 19072 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.77.193:50727
W20260806 00:57:59.796855 19348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:59.797184 19348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:59.797273 19348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:59.801468 19348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:59.801559 19348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:59.801653 19348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:57:59.806650 19348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:46515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:57:59.807917 19348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:57:59.809386 19348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:57:59.819489 19354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:57:59.819481 19353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:59.819754 19348 server_base.cc:1034] running on GCE node
W20260806 00:57:59.819770 19356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:57:59.820269 19348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:57:59.820957 19348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:57:59.822173 19348 hybrid_clock.cc:648] HybridClock initialized: now 1785977879822143 us; error 46 us; skew 500 ppm
I20260806 00:57:59.824154 19348 webserver.cc:460] Webserver started at http://127.27.77.194:45083/ using document root <none> and password file <none>
I20260806 00:57:59.824769 19348 fs_manager.cc:359] Metadata directory not provided
I20260806 00:57:59.824865 19348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:57:59.825096 19348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:57:59.827214 19348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "86423979a148491eab79c356d6021d08"
format_stamp: "Formatted at 2026-08-06 00:57:59 on dist-test-slave-sb2z"
server_key: "4df6cda6e7dc0d79e45e7306bcc89a28"
server_key_iv: "ec709f8c8bb42a172c52245f46bec45d"
server_key_version: "encryptionkey@0"
I20260806 00:57:59.827832 19348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "86423979a148491eab79c356d6021d08"
format_stamp: "Formatted at 2026-08-06 00:57:59 on dist-test-slave-sb2z"
server_key: "4df6cda6e7dc0d79e45e7306bcc89a28"
server_key_iv: "ec709f8c8bb42a172c52245f46bec45d"
server_key_version: "encryptionkey@0"
I20260806 00:57:59.831928 19348 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 00:57:59.834700 19366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:59.836174 19348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:57:59.836335 19348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "86423979a148491eab79c356d6021d08"
format_stamp: "Formatted at 2026-08-06 00:57:59 on dist-test-slave-sb2z"
server_key: "4df6cda6e7dc0d79e45e7306bcc89a28"
server_key_iv: "ec709f8c8bb42a172c52245f46bec45d"
server_key_version: "encryptionkey@0"
I20260806 00:57:59.836474 19348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:57:59.846205 19348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:57:59.846980 19348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:57:59.847205 19348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:57:59.847916 19348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:57:59.849139 19348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:57:59.849332 19348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:59.849445 19348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:57:59.849479 19348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:57:59.861096 19348 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:37573
I20260806 00:57:59.861168 19518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:37573 every 8 connection(s)
I20260806 00:57:59.862425 19348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:57:59.864760 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 19348
I20260806 00:57:59.864915 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:57:59.865293 27959 external_mini_cluster.cc:1428] Setting key 67dce78ccdf62753ce74592c96e2b002
I20260806 00:57:59.867651 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:46515
--builtin_ntp_servers=127.27.77.212:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:57:59.875242 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:59.864928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:41131 (local address 127.27.77.254:46515)
0806 00:57:59.865156 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 00:57:59.865161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:57:59.866663 (+ 1502us) server_negotiation.cc:407] Connection header received
0806 00:57:59.868573 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:57:59.868577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:57:59.868640 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:57:59.868738 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:57:59.870040 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.870635 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.872209 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.872467 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.872604 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:59.874069 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:57:59.874094 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:57:59.874101 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:57:59.874124 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:57:59.874151 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:57:59.874163 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:57:59.874253 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 00:57:59.874685 (+ 432us) server_negotiation.cc:299] Negotiation successful
0806 00:57:59.874867 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:57:59.876530 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:57:59.865559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:46515 (local address 127.27.77.194:41131)
0806 00:57:59.866485 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0806 00:57:59.866548 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 00:57:59.868313 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:57:59.868963 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:57:59.868971 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:57:59.869071 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:57:59.869826 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:59.869855 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.870798 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.870803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:59.872044 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:57:59.872057 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:57:59.872662 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:57:59.872666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:57:59.872861 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:57:59.873688 (+ 827us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:57:59.873690 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:57:59.873738 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:57:59.873816 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:57:59.874311 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:57:59.874356 (+ 45us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:57:59.874371 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 00:57:59.874651 (+ 280us) client_negotiation.cc:240] Negotiation successful
0806 00:57:59.875057 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":208,"threads_started":1}
I20260806 00:57:59.878211 19520 heartbeater.cc:347] Connected to a master server at 127.27.77.254:46515
I20260806 00:57:59.878548 19520 heartbeater.cc:464] Registering TS with master...
I20260806 00:57:59.879400 19520 heartbeater.cc:511] Master 127.27.77.254:46515 requested a full tablet report, sending...
I20260806 00:57:59.881202 19072 ts_manager.cc:194] Registered new tserver with Master: 86423979a148491eab79c356d6021d08 (127.27.77.194:37573)
I20260806 00:57:59.881875 19072 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.77.194:41131
W20260806 00:57:59.992578 19523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:57:59.992892 19523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:57:59.992935 19523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:57:59.997288 19523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:57:59.997355 19523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:57:59.997416 19523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:58:00.003132 19523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:46515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:00.004566 19523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:00.006197 19523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:00.017361 19535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:00.017341 19531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:00.017416 19523 server_base.cc:1034] running on GCE node
W20260806 00:58:00.017341 19532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:00.018229 19523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:00.018891 19523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:00.020148 19523 hybrid_clock.cc:648] HybridClock initialized: now 1785977880020124 us; error 38 us; skew 500 ppm
I20260806 00:58:00.022501 19523 webserver.cc:460] Webserver started at http://127.27.77.195:39653/ using document root <none> and password file <none>
I20260806 00:58:00.023178 19523 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:00.023280 19523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:00.023563 19523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:00.025763 19523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "dcb076977bc3445fade650745ce99348"
format_stamp: "Formatted at 2026-08-06 00:58:00 on dist-test-slave-sb2z"
server_key: "7bf365a27c89432030eca9ec95fb604a"
server_key_iv: "dbcc85fc056626d6dd2172b2ef9ef0a8"
server_key_version: "encryptionkey@0"
I20260806 00:58:00.026378 19523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "dcb076977bc3445fade650745ce99348"
format_stamp: "Formatted at 2026-08-06 00:58:00 on dist-test-slave-sb2z"
server_key: "7bf365a27c89432030eca9ec95fb604a"
server_key_iv: "dbcc85fc056626d6dd2172b2ef9ef0a8"
server_key_version: "encryptionkey@0"
I20260806 00:58:00.030393 19523 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:58:00.032970 19543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:00.034262 19523 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:58:00.034406 19523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "dcb076977bc3445fade650745ce99348"
format_stamp: "Formatted at 2026-08-06 00:58:00 on dist-test-slave-sb2z"
server_key: "7bf365a27c89432030eca9ec95fb604a"
server_key_iv: "dbcc85fc056626d6dd2172b2ef9ef0a8"
server_key_version: "encryptionkey@0"
I20260806 00:58:00.034528 19523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:00.053568 19523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:00.054353 19523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:00.054582 19523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:00.055249 19523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:58:00.056321 19523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:58:00.056397 19523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:00.056475 19523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:58:00.056521 19523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:00.066793 19523 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:39023
I20260806 00:58:00.066828 19707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:39023 every 8 connection(s)
I20260806 00:58:00.067853 19523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:58:00.077498 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 19523
I20260806 00:58:00.077857 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:58:00.078414 27959 external_mini_cluster.cc:1428] Setting key 51d94f8856a3690a1ac683c6bfd14a60
I20260806 00:58:00.079120 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:00.069831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:56907 (local address 127.27.77.254:46515)
0806 00:58:00.069990 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:00.069994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:00.071040 (+ 1046us) server_negotiation.cc:407] Connection header received
0806 00:58:00.072476 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:00.072480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:00.072530 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:00.072608 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:00.073539 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.074099 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.075520 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.075820 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.075972 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:00.078262 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:00.078266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:00.078269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:00.078293 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:00.078317 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:00.078336 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:00.078439 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:00.078830 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 00:58:00.078938 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 00:58:00.080217 19712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:00.070207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:46515 (local address 127.27.77.195:56907)
0806 00:58:00.070887 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:00.070929 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:00.072251 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:00.072775 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:00.072781 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:00.072862 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:00.073392 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:00.073406 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.074249 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.074253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:00.075315 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:00.075326 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.076065 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.076069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:00.076255 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:00.077364 (+ 1109us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:58:00.077367 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:00.077463 (+ 96us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:58:00.077905 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:00.078521 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:00.078533 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:00.078541 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 00:58:00.078761 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 00:58:00.079117 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":118,"threads_started":1}
I20260806 00:58:00.081470 19708 heartbeater.cc:347] Connected to a master server at 127.27.77.254:46515
I20260806 00:58:00.081799 19708 heartbeater.cc:464] Registering TS with master...
I20260806 00:58:00.082383 19708 heartbeater.cc:511] Master 127.27.77.254:46515 requested a full tablet report, sending...
I20260806 00:58:00.083844 19072 ts_manager.cc:194] Registered new tserver with Master: dcb076977bc3445fade650745ce99348 (127.27.77.195:39023)
I20260806 00:58:00.084468 19072 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.77.195:56907
I20260806 00:58:00.093482 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:58:00.099512 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:00.095753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54848 (local address 127.27.77.254:46515)
0806 00:58:00.095937 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:00.095941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:00.096085 (+ 144us) server_negotiation.cc:407] Connection header received
0806 00:58:00.096184 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:00.096186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:00.096235 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:00.096325 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:00.096591 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.097153 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.097968 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.098216 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.098306 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:00.099114 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:00.099117 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:00.099119 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:00.099131 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:00.099149 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:00.099157 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:00.099220 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:00.099317 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 00:58:00.099368 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 00:58:00.102739 19072 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:58:00.106462 19072 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:58:00.122303 19730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:00.116668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34166 (local address 127.27.77.194:37573)
0806 00:58:00.117227 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:00.117241 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:00.117268 (+ 27us) server_negotiation.cc:407] Connection header received
0806 00:58:00.117351 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:00.117356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:00.117611 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:00.117796 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:00.118111 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.119161 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.121001 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.121609 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.121743 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:00.121959 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:00.122022 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 00:58:00.122126 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":162,"threads_started":1}
I20260806 00:58:00.122293 19725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:00.115913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:37573 (local address 127.0.0.1:34166)
0806 00:58:00.116608 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:00.116665 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:00.116797 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:00.117785 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:00.117791 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:00.117813 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:00.117967 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:00.117977 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.119271 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.119276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:00.120837 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:00.120847 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.121742 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.121746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:00.121856 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:00.121893 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 00:58:00.121997 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 00:58:00.122129 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":155,"threads_started":1}
I20260806 00:58:00.122658 19726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:00.115997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:39023 (local address 127.0.0.1:56720)
0806 00:58:00.117137 (+ 1140us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:00.117151 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:00.117246 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:00.118006 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:00.118009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:00.118025 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:00.118176 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:00.118184 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.119711 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.119714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:00.120867 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:00.120877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.122228 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.122232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:00.122342 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:00.122365 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:58:00.122434 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 00:58:00.122500 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1057,"thread_start_us":121,"threads_started":1}
I20260806 00:58:00.123268 19729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:00.116177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56720 (local address 127.27.77.195:39023)
0806 00:58:00.116942 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:00.116970 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:00.117365 (+ 395us) server_negotiation.cc:407] Connection header received
0806 00:58:00.117454 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:00.117459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:00.117656 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:00.117822 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:00.118313 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.119569 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.121067 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.121864 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.121976 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:00.122162 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:00.122928 (+ 766us) server_negotiation.cc:299] Negotiation successful
0806 00:58:00.123089 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":132,"threads_started":1}
I20260806 00:58:00.123853 19731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:00.117200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:36007 (local address 127.0.0.1:56428)
0806 00:58:00.117442 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:00.117456 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:00.117541 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:00.118591 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:00.118595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:00.118614 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:00.118693 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:00.118699 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.120597 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.120600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:00.121794 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:00.121806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.123421 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.123425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:00.123558 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:00.123584 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:58:00.123659 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 00:58:00.123730 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":108,"threads_started":1}
I20260806 00:58:00.124312 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:00.117669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56428 (local address 127.27.77.193:36007)
0806 00:58:00.118118 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:00.118124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:00.118146 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:58:00.118215 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:00.118219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:00.118417 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:00.118604 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:00.118826 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.120477 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.121954 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.122774 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.123149 (+ 375us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:00.123346 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:00.124029 (+ 683us) server_negotiation.cc:299] Negotiation successful
0806 00:58:00.124152 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":120,"threads_started":1}
I20260806 00:58:00.125841 19429 tablet_service.cc:1467] Processing CreateTablet for tablet c658224ffd8947f7b4ba10c73cb768c1 (DEFAULT_TABLE table=test-table [id=3275c7e2ec4f4e7f9fdf81d9c7d2d392]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:58:00.125834 19613 tablet_service.cc:1467] Processing CreateTablet for tablet c658224ffd8947f7b4ba10c73cb768c1 (DEFAULT_TABLE table=test-table [id=3275c7e2ec4f4e7f9fdf81d9c7d2d392]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:58:00.126845 19429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c658224ffd8947f7b4ba10c73cb768c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:00.126899 19613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c658224ffd8947f7b4ba10c73cb768c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:00.129247 19249 tablet_service.cc:1467] Processing CreateTablet for tablet c658224ffd8947f7b4ba10c73cb768c1 (DEFAULT_TABLE table=test-table [id=3275c7e2ec4f4e7f9fdf81d9c7d2d392]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:58:00.130524 19249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c658224ffd8947f7b4ba10c73cb768c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:00.133919 19733 tablet_bootstrap.cc:492] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348: Bootstrap starting.
I20260806 00:58:00.135464 19735 tablet_bootstrap.cc:492] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08: Bootstrap starting.
I20260806 00:58:00.136564 19733 tablet_bootstrap.cc:654] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:00.137563 19733 log.cc:824] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:00.137859 19735 tablet_bootstrap.cc:654] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:00.138140 19736 tablet_bootstrap.cc:492] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426: Bootstrap starting.
I20260806 00:58:00.138816 19735 log.cc:824] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:00.140837 19735 tablet_bootstrap.cc:492] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08: No bootstrap required, opened a new log
I20260806 00:58:00.140837 19733 tablet_bootstrap.cc:492] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348: No bootstrap required, opened a new log
I20260806 00:58:00.141049 19736 tablet_bootstrap.cc:654] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:00.141064 19735 ts_tablet_manager.cc:1397] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:58:00.141064 19733 ts_tablet_manager.cc:1397] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:58:00.141791 19736 log.cc:824] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:00.143684 19736 tablet_bootstrap.cc:492] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426: No bootstrap required, opened a new log
I20260806 00:58:00.143895 19736 ts_tablet_manager.cc:1397] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:58:00.145610 19733 raft_consensus.cc:348] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86423979a148491eab79c356d6021d08" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37573 } } peers { permanent_uuid: "dcb076977bc3445fade650745ce99348" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39023 } } peers { permanent_uuid: "349154c3a8564d549fc4bac1fd3ea426" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36007 } }
I20260806 00:58:00.145610 19735 raft_consensus.cc:348] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86423979a148491eab79c356d6021d08" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37573 } } peers { permanent_uuid: "dcb076977bc3445fade650745ce99348" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39023 } } peers { permanent_uuid: "349154c3a8564d549fc4bac1fd3ea426" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36007 } }
I20260806 00:58:00.145895 19735 raft_consensus.cc:374] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:00.145895 19733 raft_consensus.cc:374] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:00.145947 19733 raft_consensus.cc:729] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcb076977bc3445fade650745ce99348, State: Initialized, Role: FOLLOWER
I20260806 00:58:00.145947 19735 raft_consensus.cc:729] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86423979a148491eab79c356d6021d08, State: Initialized, Role: FOLLOWER
I20260806 00:58:00.146433 19735 consensus_queue.cc:260] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86423979a148491eab79c356d6021d08" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37573 } } peers { permanent_uuid: "dcb076977bc3445fade650745ce99348" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39023 } } peers { permanent_uuid: "349154c3a8564d549fc4bac1fd3ea426" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36007 } }
I20260806 00:58:00.146433 19733 consensus_queue.cc:260] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86423979a148491eab79c356d6021d08" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37573 } } peers { permanent_uuid: "dcb076977bc3445fade650745ce99348" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39023 } } peers { permanent_uuid: "349154c3a8564d549fc4bac1fd3ea426" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36007 } }
I20260806 00:58:00.147401 19520 heartbeater.cc:503] Master 127.27.77.254:46515 was elected leader, sending a full tablet report...
I20260806 00:58:00.147504 19733 ts_tablet_manager.cc:1428] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 00:58:00.147505 19735 ts_tablet_manager.cc:1428] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 00:58:00.147805 19708 heartbeater.cc:503] Master 127.27.77.254:46515 was elected leader, sending a full tablet report...
I20260806 00:58:00.148393 19736 raft_consensus.cc:348] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86423979a148491eab79c356d6021d08" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37573 } } peers { permanent_uuid: "dcb076977bc3445fade650745ce99348" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39023 } } peers { permanent_uuid: "349154c3a8564d549fc4bac1fd3ea426" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36007 } }
I20260806 00:58:00.148600 19736 raft_consensus.cc:374] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:00.148676 19736 raft_consensus.cc:729] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349154c3a8564d549fc4bac1fd3ea426, State: Initialized, Role: FOLLOWER
I20260806 00:58:00.150906 19736 consensus_queue.cc:260] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86423979a148491eab79c356d6021d08" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37573 } } peers { permanent_uuid: "dcb076977bc3445fade650745ce99348" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39023 } } peers { permanent_uuid: "349154c3a8564d549fc4bac1fd3ea426" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36007 } }
I20260806 00:58:00.151731 19339 heartbeater.cc:503] Master 127.27.77.254:46515 was elected leader, sending a full tablet report...
I20260806 00:58:00.152030 19736 ts_tablet_manager.cc:1428] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260806 00:58:00.166597 19741 raft_consensus.cc:482] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260806 00:58:00.166821 19340 tablet.cc:2369] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:58:00.166868 19741 raft_consensus.cc:504] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86423979a148491eab79c356d6021d08" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37573 } } peers { permanent_uuid: "dcb076977bc3445fade650745ce99348" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39023 } } peers { permanent_uuid: "349154c3a8564d549fc4bac1fd3ea426" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36007 } }
I20260806 00:58:00.168272 19741 leader_election.cc:288] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dcb076977bc3445fade650745ce99348 (127.27.77.195:39023), 349154c3a8564d549fc4bac1fd3ea426 (127.27.77.193:36007)
I20260806 00:58:00.173717 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:00.169000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:46765 (local address 127.27.77.193:36007)
0806 00:58:00.169181 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:00.169188 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:00.169451 (+ 263us) server_negotiation.cc:407] Connection header received
0806 00:58:00.169509 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:00.169511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:00.169601 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:00.169692 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:00.169993 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.170671 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.172281 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.172900 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.173044 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:00.173097 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:00.173487 (+ 390us) server_negotiation.cc:299] Negotiation successful
0806 00:58:00.173572 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 00:58:00.173717 19729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:00.168715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:36505 (local address 127.27.77.195:39023)
0806 00:58:00.168915 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:00.168922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:00.168989 (+ 67us) server_negotiation.cc:407] Connection header received
0806 00:58:00.169070 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:00.169074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:00.169130 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:00.169224 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:00.169610 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.170390 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.172272 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.172907 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.173044 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:00.173097 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:00.173505 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 00:58:00.173580 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 00:58:00.173741 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:00.168495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:39023 (local address 127.27.77.194:36505)
0806 00:58:00.168673 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:00.168689 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:00.168819 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:00.169269 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:00.169273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:00.169298 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:00.169464 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:00.169473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.170537 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.170546 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:00.172087 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:00.172101 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.173092 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.173094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:00.173304 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:00.173348 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 00:58:00.173448 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 00:58:00.173571 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":110}
I20260806 00:58:00.173753 19748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:00.168826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:36007 (local address 127.27.77.194:46765)
0806 00:58:00.169286 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:00.169301 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:00.169405 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:00.169739 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:00.169742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:00.169760 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:00.169862 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:00.169866 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.170786 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.170790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:00.172143 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:00.172155 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.173089 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.173094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:00.173299 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:00.173356 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 00:58:00.173448 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 00:58:00.173571 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":83,"thread_start_us":173,"threads_started":1}
I20260806 00:58:00.174613 19640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c658224ffd8947f7b4ba10c73cb768c1" candidate_uuid: "86423979a148491eab79c356d6021d08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcb076977bc3445fade650745ce99348" is_pre_election: true
I20260806 00:58:00.174647 19277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c658224ffd8947f7b4ba10c73cb768c1" candidate_uuid: "86423979a148491eab79c356d6021d08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349154c3a8564d549fc4bac1fd3ea426" is_pre_election: true
I20260806 00:58:00.174970 19640 raft_consensus.cc:2454] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86423979a148491eab79c356d6021d08 in term 0.
I20260806 00:58:00.174978 19277 raft_consensus.cc:2454] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86423979a148491eab79c356d6021d08 in term 0.
I20260806 00:58:00.175547 19370 leader_election.cc:302] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349154c3a8564d549fc4bac1fd3ea426, 86423979a148491eab79c356d6021d08; no voters:
I20260806 00:58:00.175825 19741 raft_consensus.cc:2781] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:58:00.175941 19741 raft_consensus.cc:482] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:58:00.176017 19741 raft_consensus.cc:3037] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:00.177239 19741 raft_consensus.cc:504] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86423979a148491eab79c356d6021d08" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37573 } } peers { permanent_uuid: "dcb076977bc3445fade650745ce99348" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39023 } } peers { permanent_uuid: "349154c3a8564d549fc4bac1fd3ea426" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36007 } }
I20260806 00:58:00.177650 19741 leader_election.cc:288] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [CANDIDATE]: Term 1 election: Requested vote from peers dcb076977bc3445fade650745ce99348 (127.27.77.195:39023), 349154c3a8564d549fc4bac1fd3ea426 (127.27.77.193:36007)
I20260806 00:58:00.178179 19640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c658224ffd8947f7b4ba10c73cb768c1" candidate_uuid: "86423979a148491eab79c356d6021d08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcb076977bc3445fade650745ce99348"
I20260806 00:58:00.178253 19277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c658224ffd8947f7b4ba10c73cb768c1" candidate_uuid: "86423979a148491eab79c356d6021d08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "349154c3a8564d549fc4bac1fd3ea426"
I20260806 00:58:00.178330 19640 raft_consensus.cc:3037] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:00.178372 19277 raft_consensus.cc:3037] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:00.179750 19277 raft_consensus.cc:2454] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86423979a148491eab79c356d6021d08 in term 1.
I20260806 00:58:00.179750 19640 raft_consensus.cc:2454] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86423979a148491eab79c356d6021d08 in term 1.
I20260806 00:58:00.180083 19370 leader_election.cc:302] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86423979a148491eab79c356d6021d08, dcb076977bc3445fade650745ce99348; no voters:
I20260806 00:58:00.180289 19741 raft_consensus.cc:2781] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:00.180511 19741 raft_consensus.cc:686] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [term 1 LEADER]: Becoming Leader. State: Replica: 86423979a148491eab79c356d6021d08, State: Running, Role: LEADER
I20260806 00:58:00.180879 19741 consensus_queue.cc:237] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86423979a148491eab79c356d6021d08" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37573 } } peers { permanent_uuid: "dcb076977bc3445fade650745ce99348" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39023 } } peers { permanent_uuid: "349154c3a8564d549fc4bac1fd3ea426" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36007 } }
I20260806 00:58:00.184607 19066 catalog_manager.cc:5495] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86423979a148491eab79c356d6021d08 (127.27.77.194). New cstate: current_term: 1 leader_uuid: "86423979a148491eab79c356d6021d08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86423979a148491eab79c356d6021d08" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 37573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dcb076977bc3445fade650745ce99348" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349154c3a8564d549fc4bac1fd3ea426" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36007 } health_report { overall_health: UNKNOWN } } }
I20260806 00:58:00.203828 19730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:00.200159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34182 (local address 127.27.77.194:37573)
0806 00:58:00.200294 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:00.200298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:00.200312 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:58:00.200393 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:00.200395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:00.200451 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:00.200556 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:58:00.200801 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.201356 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.202280 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:00.202501 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:00.202585 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:00.202929 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:58:00.203397 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:58:00.203494 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:00.203645 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 00:58:00.203700 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:58:00.210445 19640 raft_consensus.cc:1261] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348 [term 1 FOLLOWER]: Refusing update from remote peer 86423979a148491eab79c356d6021d08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:58:00.210445 19277 raft_consensus.cc:1261] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426 [term 1 FOLLOWER]: Refusing update from remote peer 86423979a148491eab79c356d6021d08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:58:00.211280 19741 consensus_queue.cc:1035] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcb076977bc3445fade650745ce99348" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:58:00.211589 19749 consensus_queue.cc:1035] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "349154c3a8564d549fc4bac1fd3ea426" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 36007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:58:00.220621 19757 mvcc.cc:205] Tried to move back new op lower bound from 7315365397333495808 to 7315365397223727104. Current Snapshot: MvccSnapshot[applied={T|T < 7315365397333495808}]
I20260806 00:58:00.220865 19754 mvcc.cc:205] Tried to move back new op lower bound from 7315365397333495808 to 7315365397223727104. Current Snapshot: MvccSnapshot[applied={T|T < 7315365397333495808}]
I20260806 00:58:00.229251 19072 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:58:00.229436 19072 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:58:00.232188 19072 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a7478eee528437ca4e237fee55e94c8: Sending DeleteTablet for 3 replicas of tablet c658224ffd8947f7b4ba10c73cb768c1
I20260806 00:58:00.233161 19429 tablet_service.cc:1514] Processing DeleteTablet for tablet c658224ffd8947f7b4ba10c73cb768c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:58:00 UTC) from {username='slave'} at 127.0.0.1:34166
I20260806 00:58:00.233270 19613 tablet_service.cc:1514] Processing DeleteTablet for tablet c658224ffd8947f7b4ba10c73cb768c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:58:00 UTC) from {username='slave'} at 127.0.0.1:56720
I20260806 00:58:00.233424 19249 tablet_service.cc:1514] Processing DeleteTablet for tablet c658224ffd8947f7b4ba10c73cb768c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:58:00 UTC) from {username='slave'} at 127.0.0.1:56428
I20260806 00:58:00.233927 19770 tablet_replica.cc:330] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426: stopping tablet replica
I20260806 00:58:00.233958 19769 tablet_replica.cc:330] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348: stopping tablet replica
I20260806 00:58:00.234280 19770 raft_consensus.cc:2229] T c658224ffd8947f7b4ba10c73cb768c1 P 349154c3a8564d549fc4bac1fd3ea426 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:58:00.234293 19769 raft_consensus.cc:2229] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:58:00.234673 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 19149
I20260806 00:58:00.234697 19769 raft_consensus.cc:2258] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:58:00.236534 19769 ts_tablet_manager.cc:1905] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:58:00.238341 19768 tablet_replica.cc:330] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08: stopping tablet replica
I20260806 00:58:00.238674 19768 raft_consensus.cc:2229] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:58:00.239035 19768 raft_consensus.cc:2258] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:58:00.241354 19769 ts_tablet_manager.cc:1918] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:58:00.241457 19769 log.cc:1196] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/c658224ffd8947f7b4ba10c73cb768c1
I20260806 00:58:00.241776 19769 ts_tablet_manager.cc:1939] T c658224ffd8947f7b4ba10c73cb768c1 P dcb076977bc3445fade650745ce99348: Deleting consensus metadata
I20260806 00:58:00.243566 19046 catalog_manager.cc:4841] TS dcb076977bc3445fade650745ce99348 (127.27.77.195:39023): tablet c658224ffd8947f7b4ba10c73cb768c1 (table test-table [id=3275c7e2ec4f4e7f9fdf81d9c7d2d392]) successfully deleted
I20260806 00:58:00.244676 19768 ts_tablet_manager.cc:1905] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:58:00.246925 19768 ts_tablet_manager.cc:1918] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:58:00.247061 19768 log.cc:1196] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/c658224ffd8947f7b4ba10c73cb768c1
I20260806 00:58:00.247390 19768 ts_tablet_manager.cc:1939] T c658224ffd8947f7b4ba10c73cb768c1 P 86423979a148491eab79c356d6021d08: Deleting consensus metadata
I20260806 00:58:00.248685 19044 catalog_manager.cc:4841] TS 86423979a148491eab79c356d6021d08 (127.27.77.194:37573): tablet c658224ffd8947f7b4ba10c73cb768c1 (table test-table [id=3275c7e2ec4f4e7f9fdf81d9c7d2d392]) successfully deleted
W20260806 00:58:00.254240 19046 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:36007 (error 108)
I20260806 00:58:00.254273 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 19348
I20260806 00:58:00.262066 19726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:00.261695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:36007 (local address 127.0.0.1:56430)
0806 00:58:00.261854 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:00.261948 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:36007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 00:58:00.262326 19046 catalog_manager.cc:4584] TS 349154c3a8564d549fc4bac1fd3ea426 (127.27.77.193:36007): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c658224ffd8947f7b4ba10c73cb768c1: Network error: Client connection negotiation failed: client connection to 127.27.77.193:36007: connect: Connection refused (error 111)
I20260806 00:58:00.267899 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 19523
I20260806 00:58:00.278998 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 19023
2026-08-06T00:58:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1114 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:00 dist-test-slave-sb2z krb5kdc[19774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:00 dist-test-slave-sb2z krb5kdc[19774](info): set up 2 sockets
Aug 06 00:58:00 dist-test-slave-sb2z krb5kdc[19774](info): commencing operation
krb5kdc: starting...
W20260806 00:58:01.511724 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:01 dist-test-slave-sb2z krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:58:01.650821 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:33339
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:46307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:33339
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:01.768975 19792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:01.769276 19792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:01.769320 19792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:01.773550 19792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:01.773658 19792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:01.773679 19792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:01.773697 19792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:01.773713 19792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:01.779270 19792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:46307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:33339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:33339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:01.780484 19792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:01.781590 19792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:01.790848 19804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:01.790848 19799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:01.790979 19802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:01.791409 19792 server_base.cc:1034] running on GCE node
I20260806 00:58:01.792140 19792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:01.793426 19792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:01.794646 19792 hybrid_clock.cc:648] HybridClock initialized: now 1785977881794623 us; error 41 us; skew 500 ppm
Aug 06 00:58:01 dist-test-slave-sb2z krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977881, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:01.800926 19792 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:58:01.801963 19792 webserver.cc:460] Webserver started at http://127.27.77.254:41351/ using document root <none> and password file <none>
I20260806 00:58:01.802543 19792 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:01.802606 19792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:01.802783 19792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:01.804998 19792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "8e318263622e4087afdfb82b835852d3"
format_stamp: "Formatted at 2026-08-06 00:58:01 on dist-test-slave-sb2z"
server_key: "d56c97bb57f61f2e6d43c69017d071e6"
server_key_iv: "12b04dc1f78f6abd6832aa9688831398"
server_key_version: "encryptionkey@0"
I20260806 00:58:01.805541 19792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "8e318263622e4087afdfb82b835852d3"
format_stamp: "Formatted at 2026-08-06 00:58:01 on dist-test-slave-sb2z"
server_key: "d56c97bb57f61f2e6d43c69017d071e6"
server_key_iv: "12b04dc1f78f6abd6832aa9688831398"
server_key_version: "encryptionkey@0"
I20260806 00:58:01.809656 19792 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:58:01.812273 19812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:01.813676 19792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:01.813789 19792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "8e318263622e4087afdfb82b835852d3"
format_stamp: "Formatted at 2026-08-06 00:58:01 on dist-test-slave-sb2z"
server_key: "d56c97bb57f61f2e6d43c69017d071e6"
server_key_iv: "12b04dc1f78f6abd6832aa9688831398"
server_key_version: "encryptionkey@0"
I20260806 00:58:01.813889 19792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:01.832839 19792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:01.836442 19792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:01.836620 19792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:01.844663 19792 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:33339
I20260806 00:58:01.844677 19888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:33339 every 8 connection(s)
I20260806 00:58:01.845774 19792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:01.849025 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 19792
I20260806 00:58:01.849123 19890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:01.849195 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:01.849534 27959 external_mini_cluster.cc:1428] Setting key ff46bd917ddc35044769ecba3dfa5bcc
I20260806 00:58:01.856281 19890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3: Bootstrap starting.
I20260806 00:58:01.858947 19890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3: Neither blocks nor log segments found. Creating new log.
Aug 06 00:58:01 dist-test-slave-sb2z krb5kdc[19774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:58:01.860107 19890 log.cc:824] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:01.862668 19890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3: No bootstrap required, opened a new log
I20260806 00:58:01.865854 19894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:01.851107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59990 (local address 127.27.77.254:33339)
0806 00:58:01.851680 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:01.851699 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:01.851738 (+ 39us) server_negotiation.cc:407] Connection header received
0806 00:58:01.852492 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:01.852513 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:01.852881 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:01.853257 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:01.854181 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:01.855270 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:01.856330 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:01.856938 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:01.857135 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:01.860382 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:01.860388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:01.860404 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:01.860437 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:01.862631 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:01.863235 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:01.863240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:01.863246 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:01.863366 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:01.863724 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:01.863727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:01.863728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:01.864193 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:01.864352 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:01.864714 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 00:58:01.865095 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":148,"threads_started":1}
I20260806 00:58:01.867976 19890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e318263622e4087afdfb82b835852d3" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33339 } }
I20260806 00:58:01.868286 19890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:01.868345 19890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e318263622e4087afdfb82b835852d3, State: Initialized, Role: FOLLOWER
I20260806 00:58:01.868885 19890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e318263622e4087afdfb82b835852d3" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33339 } }
I20260806 00:58:01.869056 19890 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:01.869107 19890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:01.869184 19890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:01.870179 19890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e318263622e4087afdfb82b835852d3" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33339 } }
I20260806 00:58:01.870500 19890 leader_election.cc:302] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e318263622e4087afdfb82b835852d3; no voters:
I20260806 00:58:01.870818 19890 leader_election.cc:288] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:01.871014 19896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:01.871286 19896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [term 1 LEADER]: Becoming Leader. State: Replica: 8e318263622e4087afdfb82b835852d3, State: Running, Role: LEADER
I20260806 00:58:01.871659 19896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e318263622e4087afdfb82b835852d3" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33339 } }
I20260806 00:58:01.872318 19890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:01.873519 19898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e318263622e4087afdfb82b835852d3. Latest consensus state: current_term: 1 leader_uuid: "8e318263622e4087afdfb82b835852d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e318263622e4087afdfb82b835852d3" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33339 } } }
I20260806 00:58:01.873672 19898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:01.874122 19897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e318263622e4087afdfb82b835852d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e318263622e4087afdfb82b835852d3" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 33339 } } }
I20260806 00:58:01.874532 19897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:01.875108 19907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:01.878190 19907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:01.883899 19907 catalog_manager.cc:1347] Generated new cluster ID: 3e4d170bdd934b23bdaeb436baa424f8
I20260806 00:58:01.883986 19907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:01.889788 19907 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:01.891057 19907 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:01.898960 19907 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3: Generated new TSK 0
I20260806 00:58:01.899916 19907 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:58:01.940927 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33339
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:46307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:02.064060 19924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:02.064338 19924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:02.064384 19924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:02.069020 19924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:02.069106 19924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:02.069233 19924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:58:02.075801 19924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:46307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:02.077122 19924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:02.078287 19924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:02.089089 19934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:02.089133 19932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:02.089234 19931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:02.090437 19924 server_base.cc:1034] running on GCE node
I20260806 00:58:02.090932 19924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:02.091718 19924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:02.092931 19924 hybrid_clock.cc:648] HybridClock initialized: now 1785977882092901 us; error 41 us; skew 500 ppm
Aug 06 00:58:02 dist-test-slave-sb2z krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977882, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:02.096470 19924 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:58:02.097582 19924 webserver.cc:460] Webserver started at http://127.27.77.193:46109/ using document root <none> and password file <none>
I20260806 00:58:02.098240 19924 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:02.098311 19924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:02.098608 19924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:02.101076 19924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "206b092bd0c94ae6b03a6428cf272ed9"
format_stamp: "Formatted at 2026-08-06 00:58:02 on dist-test-slave-sb2z"
server_key: "9e6c1ddbedaf665c1af7426de1c0400a"
server_key_iv: "3adec2ffa1735078792a397b81db2552"
server_key_version: "encryptionkey@0"
I20260806 00:58:02.101734 19924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "206b092bd0c94ae6b03a6428cf272ed9"
format_stamp: "Formatted at 2026-08-06 00:58:02 on dist-test-slave-sb2z"
server_key: "9e6c1ddbedaf665c1af7426de1c0400a"
server_key_iv: "3adec2ffa1735078792a397b81db2552"
server_key_version: "encryptionkey@0"
I20260806 00:58:02.106244 19924 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:58:02.109143 19943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:02.110634 19924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 00:58:02.110829 19924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "206b092bd0c94ae6b03a6428cf272ed9"
format_stamp: "Formatted at 2026-08-06 00:58:02 on dist-test-slave-sb2z"
server_key: "9e6c1ddbedaf665c1af7426de1c0400a"
server_key_iv: "3adec2ffa1735078792a397b81db2552"
server_key_version: "encryptionkey@0"
I20260806 00:58:02.110991 19924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:02.138581 19924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:02.142470 19924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:02.142707 19924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:02.143388 19924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:58:02.144388 19924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:58:02.144466 19924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:02.144541 19924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:58:02.144591 19924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:02.155782 19924 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:44197
I20260806 00:58:02.155822 20104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:44197 every 8 connection(s)
I20260806 00:58:02.156809 19924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:58:02.159880 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 19924
I20260806 00:58:02.160051 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:58:02.160418 27959 external_mini_cluster.cc:1428] Setting key b44637f1c7854c7630dd6847cbea6a20
Aug 06 00:58:02 dist-test-slave-sb2z krb5kdc[19774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977882, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
I20260806 00:58:02.173635 19894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.158922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:58697 (local address 127.27.77.254:33339)
0806 00:58:02.159070 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:02.159081 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:02.160018 (+ 937us) server_negotiation.cc:407] Connection header received
0806 00:58:02.161788 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:02.161791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:02.161848 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:02.161925 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:02.163276 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.163911 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.165194 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.165439 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.165562 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.168804 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:02.168809 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:02.168812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:02.168851 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:02.170871 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:02.171523 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:02.171555 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:02.171558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:02.171682 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:02.172137 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:02.172143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:02.172146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:02.172445 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:02.172582 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:02.173303 (+ 721us) server_negotiation.cc:299] Negotiation successful
0806 00:58:02.173480 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:58:02.174906 20108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.159249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:33339 (local address 127.27.77.193:58697)
0806 00:58:02.159837 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:02.159886 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:02.161545 (+ 1659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:02.162121 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:02.162129 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:02.162544 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:02.163075 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.163090 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.164073 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.164076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.165047 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.165057 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.165616 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.165621 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.165885 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.166590 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:02.166617 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:02.168602 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:02.171067 (+ 2465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:02.171073 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:02.171088 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:02.171393 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:02.171806 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:02.171809 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:02.171811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:02.172015 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:02.172606 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:02.172613 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:02.172888 (+ 275us) client_negotiation.cc:769] Sending connection context
0806 00:58:02.173275 (+ 387us) client_negotiation.cc:240] Negotiation successful
0806 00:58:02.173700 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":127,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
I20260806 00:58:02.176807 20106 heartbeater.cc:347] Connected to a master server at 127.27.77.254:33339
I20260806 00:58:02.177119 20106 heartbeater.cc:464] Registering TS with master...
I20260806 00:58:02.177703 20106 heartbeater.cc:511] Master 127.27.77.254:33339 requested a full tablet report, sending...
I20260806 00:58:02.179497 19828 ts_manager.cc:194] Registered new tserver with Master: 206b092bd0c94ae6b03a6428cf272ed9 (127.27.77.193:44197)
I20260806 00:58:02.180815 19828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:58697
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:58:02.195240 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33339
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:46307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:02.320372 20111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:02.320642 20111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:02.320712 20111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:02.324932 20111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:02.324996 20111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:02.325098 20111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:58:02.330454 20111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:46307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:02.331574 20111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:02.332602 20111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:02.342607 20123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:02.342659 20120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:02.342624 20121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:02.343065 20111 server_base.cc:1034] running on GCE node
I20260806 00:58:02.343483 20111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:02.344223 20111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:02.345422 20111 hybrid_clock.cc:648] HybridClock initialized: now 1785977882345403 us; error 40 us; skew 500 ppm
Aug 06 00:58:02 dist-test-slave-sb2z krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977882, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:02.348449 20111 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:58:02.349439 20111 webserver.cc:460] Webserver started at http://127.27.77.194:43241/ using document root <none> and password file <none>
I20260806 00:58:02.349984 20111 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:02.350046 20111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:02.350219 20111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:02.352309 20111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "a3e540d6e7584fda8d56a1f8af38b25d"
format_stamp: "Formatted at 2026-08-06 00:58:02 on dist-test-slave-sb2z"
server_key: "c1067737c355a2b58e75b6371d4ec6a8"
server_key_iv: "5306c7a7d69ec89a690d7c92ad2dec8c"
server_key_version: "encryptionkey@0"
I20260806 00:58:02.352833 20111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "a3e540d6e7584fda8d56a1f8af38b25d"
format_stamp: "Formatted at 2026-08-06 00:58:02 on dist-test-slave-sb2z"
server_key: "c1067737c355a2b58e75b6371d4ec6a8"
server_key_iv: "5306c7a7d69ec89a690d7c92ad2dec8c"
server_key_version: "encryptionkey@0"
I20260806 00:58:02.356771 20111 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:58:02.359368 20134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:02.360828 20111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:02.360996 20111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "a3e540d6e7584fda8d56a1f8af38b25d"
format_stamp: "Formatted at 2026-08-06 00:58:02 on dist-test-slave-sb2z"
server_key: "c1067737c355a2b58e75b6371d4ec6a8"
server_key_iv: "5306c7a7d69ec89a690d7c92ad2dec8c"
server_key_version: "encryptionkey@0"
I20260806 00:58:02.361124 20111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:02.378131 20111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:02.381812 20111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:02.382046 20111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:02.382679 20111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:58:02.383657 20111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:58:02.383733 20111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:02.383805 20111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:58:02.383857 20111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:02.393586 20111 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:36099
I20260806 00:58:02.393646 20313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:36099 every 8 connection(s)
I20260806 00:58:02.394680 20111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:58:02.404101 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 20111
I20260806 00:58:02.404251 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:58:02.404573 27959 external_mini_cluster.cc:1428] Setting key eb2c5d1de97f889fa45f9c1d3764ec82
Aug 06 00:58:02 dist-test-slave-sb2z krb5kdc[19774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977882, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:58:02.410665 19894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.396750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:34525 (local address 127.27.77.254:33339)
0806 00:58:02.396913 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:02.396917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:02.397735 (+ 818us) server_negotiation.cc:407] Connection header received
0806 00:58:02.399518 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:02.399558 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:02.399649 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:02.399758 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:02.401201 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.401782 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.403076 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.403330 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.403452 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.406475 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:02.406479 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:02.406482 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:02.406510 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:02.408388 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:02.409012 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:02.409015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:02.409017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:02.409113 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:02.409486 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:02.409490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:02.409492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:02.409692 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:02.409806 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:02.410351 (+ 545us) server_negotiation.cc:299] Negotiation successful
0806 00:58:02.410488 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:58:02.411585 20321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.397065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:33339 (local address 127.27.77.194:34525)
0806 00:58:02.397562 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:02.397610 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:02.399272 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:02.399956 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:02.399971 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:02.400378 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:02.401037 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.401054 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.401937 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.401940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.402918 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.402928 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.403515 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.403520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.403748 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.404441 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:02.404460 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:02.406308 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:02.408515 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:02.408523 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:02.408542 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:02.408878 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:02.409218 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:02.409222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:02.409224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:02.409380 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:02.409808 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:02.409817 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:02.410098 (+ 281us) client_negotiation.cc:769] Sending connection context
0806 00:58:02.410312 (+ 214us) client_negotiation.cc:240] Negotiation successful
0806 00:58:02.410664 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":116,"threads_started":1}
I20260806 00:58:02.412896 20314 heartbeater.cc:347] Connected to a master server at 127.27.77.254:33339
I20260806 00:58:02.413208 20314 heartbeater.cc:464] Registering TS with master...
I20260806 00:58:02.413722 20314 heartbeater.cc:511] Master 127.27.77.254:33339 requested a full tablet report, sending...
I20260806 00:58:02.414866 19828 ts_manager.cc:194] Registered new tserver with Master: a3e540d6e7584fda8d56a1f8af38b25d (127.27.77.194:36099)
I20260806 00:58:02.415812 19828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:34525
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:58:02.439002 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33339
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:46307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:02.562369 20327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:02.562726 20327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:02.562783 20327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:02.567731 20327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:02.567809 20327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:02.567878 20327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:58:02.574258 20327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:46307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:33339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:02.575470 20327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:02.576700 20327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:02.587119 20332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:02.587390 20327 server_base.cc:1034] running on GCE node
W20260806 00:58:02.587173 20338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:02.587119 20333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:02.587971 20327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:02.588738 20327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:02.589969 20327 hybrid_clock.cc:648] HybridClock initialized: now 1785977882589951 us; error 28 us; skew 500 ppm
Aug 06 00:58:02 dist-test-slave-sb2z krb5kdc[19774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977882, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:02.593065 20327 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:58:02.594101 20327 webserver.cc:460] Webserver started at http://127.27.77.195:37583/ using document root <none> and password file <none>
I20260806 00:58:02.594722 20327 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:02.594820 20327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:02.595247 20327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:02.597590 20327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "33ba1ca45e394c72801a3b317df426c2"
format_stamp: "Formatted at 2026-08-06 00:58:02 on dist-test-slave-sb2z"
server_key: "05157e597df32437258d676773e9e787"
server_key_iv: "115260d7fa5dc995c6953c453db168d4"
server_key_version: "encryptionkey@0"
I20260806 00:58:02.598192 20327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "33ba1ca45e394c72801a3b317df426c2"
format_stamp: "Formatted at 2026-08-06 00:58:02 on dist-test-slave-sb2z"
server_key: "05157e597df32437258d676773e9e787"
server_key_iv: "115260d7fa5dc995c6953c453db168d4"
server_key_version: "encryptionkey@0"
I20260806 00:58:02.602200 20327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:58:02.604784 20347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:02.606115 20327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:58:02.606276 20327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "33ba1ca45e394c72801a3b317df426c2"
format_stamp: "Formatted at 2026-08-06 00:58:02 on dist-test-slave-sb2z"
server_key: "05157e597df32437258d676773e9e787"
server_key_iv: "115260d7fa5dc995c6953c453db168d4"
server_key_version: "encryptionkey@0"
I20260806 00:58:02.606400 20327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:02.624764 20327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:02.628229 20327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:02.628461 20327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:02.629093 20327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:58:02.629959 20327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:58:02.630033 20327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:02.630110 20327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:58:02.630156 20327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:02.640012 20327 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:42611
I20260806 00:58:02.640035 20503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:42611 every 8 connection(s)
I20260806 00:58:02.641139 20327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:58:02.647662 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 20327
I20260806 00:58:02.647819 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:58:02.648161 27959 external_mini_cluster.cc:1428] Setting key 2f3f547357d90e1d0fa74d4d59c3cdad
Aug 06 00:58:02 dist-test-slave-sb2z krb5kdc[19774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977882, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:58:02.656252 19894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.643170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:43535 (local address 127.27.77.254:33339)
0806 00:58:02.643353 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:02.643358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:02.644199 (+ 841us) server_negotiation.cc:407] Connection header received
0806 00:58:02.645532 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:02.645535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:02.645604 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:02.645682 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:02.646876 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.647417 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.649153 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.649484 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.649605 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.652613 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:02.652617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:02.652619 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:02.652645 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:02.653971 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:02.654535 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:02.654538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:02.654539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:02.654622 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:02.654995 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:02.654998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:02.654999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:02.655210 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:02.655312 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:02.656013 (+ 701us) server_negotiation.cc:299] Negotiation successful
0806 00:58:02.656126 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 00:58:02.657102 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.643498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:33339 (local address 127.27.77.195:43535)
0806 00:58:02.644025 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:02.644078 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:02.645327 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:02.645814 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:02.645820 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:02.646207 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:02.646716 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.646731 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.647597 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.647602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.648946 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.648958 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.649670 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.649674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.649928 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.650713 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:02.650735 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:02.652423 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:02.654112 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:02.654118 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:02.654131 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:02.654408 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:02.654706 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:02.654708 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:02.654709 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:02.654854 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:02.655374 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:02.655380 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:02.655731 (+ 351us) client_negotiation.cc:769] Sending connection context
0806 00:58:02.655903 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 00:58:02.656187 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":111,"threads_started":1}
I20260806 00:58:02.658294 20505 heartbeater.cc:347] Connected to a master server at 127.27.77.254:33339
I20260806 00:58:02.658569 20505 heartbeater.cc:464] Registering TS with master...
I20260806 00:58:02.659125 20505 heartbeater.cc:511] Master 127.27.77.254:33339 requested a full tablet report, sending...
I20260806 00:58:02.660348 19828 ts_manager.cc:194] Registered new tserver with Master: 33ba1ca45e394c72801a3b317df426c2 (127.27.77.195:42611)
I20260806 00:58:02.660970 19828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:43535
I20260806 00:58:02.663017 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:58:02.674708 19894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.665550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60004 (local address 127.27.77.254:33339)
0806 00:58:02.665705 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:02.665710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:02.665905 (+ 195us) server_negotiation.cc:407] Connection header received
0806 00:58:02.666013 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:02.666016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:02.666057 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:02.666118 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:02.666827 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.667344 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.668276 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.668555 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.668664 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.670543 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:02.670546 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:02.670548 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:02.670573 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:02.672471 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:02.673104 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:02.673107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:02.673108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:02.673202 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:02.673718 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:02.673722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:02.673724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:02.673957 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:02.674137 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:02.674426 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 00:58:02.674542 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 00:58:02.678185 19828 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60004:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:58:02.681497 19828 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:58:02.698369 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.692304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:44197 (local address 127.0.0.1:37592)
0806 00:58:02.692645 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:02.692662 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:02.692767 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:02.693537 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:02.693540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:02.693558 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:02.693652 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.693658 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.694842 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.694847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.696791 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.696804 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.697692 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.697696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.697825 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.697869 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 00:58:02.698031 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 00:58:02.698190 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":246,"thread_start_us":128,"threads_started":1}
I20260806 00:58:02.698375 20523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.692542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37592 (local address 127.27.77.193:44197)
0806 00:58:02.692940 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:02.692948 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:02.692974 (+ 26us) server_negotiation.cc:407] Connection header received
0806 00:58:02.693061 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:02.693066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:02.693265 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:02.693444 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:02.693797 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.694723 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.696901 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.697549 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.697693 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.697862 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:02.698071 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 00:58:02.698185 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":82,"threads_started":1}
I20260806 00:58:02.698401 20520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.691493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:42611 (local address 127.0.0.1:41096)
0806 00:58:02.692068 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:02.692142 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:02.692338 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:02.693030 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:02.693036 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:02.693058 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:02.693218 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.693227 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.694869 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.694872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.696711 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.696725 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.697878 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.697882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.697969 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.697991 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:58:02.698047 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 00:58:02.698182 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":149,"thread_start_us":114,"threads_started":1}
I20260806 00:58:02.698954 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.691760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41096 (local address 127.27.77.195:42611)
0806 00:58:02.692463 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:02.692477 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:02.692504 (+ 27us) server_negotiation.cc:407] Connection header received
0806 00:58:02.692589 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:02.692598 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:02.692846 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:02.693384 (+ 538us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:02.693544 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.694715 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.696856 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.697736 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.698498 (+ 762us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.698667 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:02.698711 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 00:58:02.698795 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":201,"threads_started":1}
I20260806 00:58:02.699352 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.693130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55770 (local address 127.27.77.194:36099)
0806 00:58:02.693928 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:02.693938 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:02.693960 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:58:02.694032 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:02.694036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:02.694215 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:02.694362 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:02.694620 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.695571 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.696923 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.697736 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.697854 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.697993 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:02.698443 (+ 450us) server_negotiation.cc:299] Negotiation successful
0806 00:58:02.699202 (+ 759us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":133,"threads_started":1}
I20260806 00:58:02.699375 20526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.692664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:36099 (local address 127.0.0.1:55770)
0806 00:58:02.693556 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:02.693573 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:02.693670 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:02.694377 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:02.694380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:02.694402 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:02.694493 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.694499 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.695728 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.695731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.696756 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.696766 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.698069 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.698072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.698167 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.698189 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:58:02.698257 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 00:58:02.698322 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"mutex_wait_us":41,"thread_start_us":84,"threads_started":1}
I20260806 00:58:02.702227 20008 tablet_service.cc:1467] Processing CreateTablet for tablet 36f12497b3d94a54bbc31de9c3f97b75 (DEFAULT_TABLE table=test-table [id=7f51682ad54d47b7a50124374f27733d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:58:02.702319 20208 tablet_service.cc:1467] Processing CreateTablet for tablet 36f12497b3d94a54bbc31de9c3f97b75 (DEFAULT_TABLE table=test-table [id=7f51682ad54d47b7a50124374f27733d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:58:02.702319 20413 tablet_service.cc:1467] Processing CreateTablet for tablet 36f12497b3d94a54bbc31de9c3f97b75 (DEFAULT_TABLE table=test-table [id=7f51682ad54d47b7a50124374f27733d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:58:02.703451 20413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36f12497b3d94a54bbc31de9c3f97b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:02.703451 20208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36f12497b3d94a54bbc31de9c3f97b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:02.703473 20008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36f12497b3d94a54bbc31de9c3f97b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:02.711208 20528 tablet_bootstrap.cc:492] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2: Bootstrap starting.
I20260806 00:58:02.711753 20532 tablet_bootstrap.cc:492] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9: Bootstrap starting.
I20260806 00:58:02.711838 20529 tablet_bootstrap.cc:492] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d: Bootstrap starting.
I20260806 00:58:02.713943 20529 tablet_bootstrap.cc:654] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:02.713964 20528 tablet_bootstrap.cc:654] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:02.715492 20529 log.cc:824] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:02.715505 20528 log.cc:824] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:02.716864 20532 tablet_bootstrap.cc:654] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:02.717770 20532 log.cc:824] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:02.718221 20529 tablet_bootstrap.cc:492] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d: No bootstrap required, opened a new log
I20260806 00:58:02.718504 20529 ts_tablet_manager.cc:1397] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 00:58:02.718806 20528 tablet_bootstrap.cc:492] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2: No bootstrap required, opened a new log
I20260806 00:58:02.719038 20528 ts_tablet_manager.cc:1397] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260806 00:58:02.719853 20532 tablet_bootstrap.cc:492] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9: No bootstrap required, opened a new log
I20260806 00:58:02.720036 20532 ts_tablet_manager.cc:1397] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260806 00:58:02.722782 20528 raft_consensus.cc:348] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ba1ca45e394c72801a3b317df426c2" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42611 } } peers { permanent_uuid: "206b092bd0c94ae6b03a6428cf272ed9" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44197 } } peers { permanent_uuid: "a3e540d6e7584fda8d56a1f8af38b25d" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36099 } }
I20260806 00:58:02.722863 20529 raft_consensus.cc:348] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ba1ca45e394c72801a3b317df426c2" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42611 } } peers { permanent_uuid: "206b092bd0c94ae6b03a6428cf272ed9" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44197 } } peers { permanent_uuid: "a3e540d6e7584fda8d56a1f8af38b25d" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36099 } }
I20260806 00:58:02.723084 20528 raft_consensus.cc:374] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:02.723140 20528 raft_consensus.cc:729] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33ba1ca45e394c72801a3b317df426c2, State: Initialized, Role: FOLLOWER
I20260806 00:58:02.723179 20529 raft_consensus.cc:374] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:02.723232 20529 raft_consensus.cc:729] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3e540d6e7584fda8d56a1f8af38b25d, State: Initialized, Role: FOLLOWER
I20260806 00:58:02.723747 20528 consensus_queue.cc:260] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ba1ca45e394c72801a3b317df426c2" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42611 } } peers { permanent_uuid: "206b092bd0c94ae6b03a6428cf272ed9" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44197 } } peers { permanent_uuid: "a3e540d6e7584fda8d56a1f8af38b25d" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36099 } }
I20260806 00:58:02.723789 20529 consensus_queue.cc:260] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ba1ca45e394c72801a3b317df426c2" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42611 } } peers { permanent_uuid: "206b092bd0c94ae6b03a6428cf272ed9" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44197 } } peers { permanent_uuid: "a3e540d6e7584fda8d56a1f8af38b25d" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36099 } }
I20260806 00:58:02.724745 20528 ts_tablet_manager.cc:1428] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:58:02.724588 20532 raft_consensus.cc:348] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ba1ca45e394c72801a3b317df426c2" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42611 } } peers { permanent_uuid: "206b092bd0c94ae6b03a6428cf272ed9" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44197 } } peers { permanent_uuid: "a3e540d6e7584fda8d56a1f8af38b25d" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36099 } }
I20260806 00:58:02.724799 20529 ts_tablet_manager.cc:1428] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:58:02.724860 20532 raft_consensus.cc:374] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:02.724921 20532 raft_consensus.cc:729] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 206b092bd0c94ae6b03a6428cf272ed9, State: Initialized, Role: FOLLOWER
I20260806 00:58:02.725184 20505 heartbeater.cc:503] Master 127.27.77.254:33339 was elected leader, sending a full tablet report...
I20260806 00:58:02.725390 20532 consensus_queue.cc:260] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ba1ca45e394c72801a3b317df426c2" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42611 } } peers { permanent_uuid: "206b092bd0c94ae6b03a6428cf272ed9" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44197 } } peers { permanent_uuid: "a3e540d6e7584fda8d56a1f8af38b25d" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36099 } }
I20260806 00:58:02.726043 20106 heartbeater.cc:503] Master 127.27.77.254:33339 was elected leader, sending a full tablet report...
I20260806 00:58:02.726267 20532 ts_tablet_manager.cc:1428] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:58:02.726332 20314 heartbeater.cc:503] Master 127.27.77.254:33339 was elected leader, sending a full tablet report...
I20260806 00:58:02.883476 20539 raft_consensus.cc:482] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:58:02.883782 20539 raft_consensus.cc:504] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ba1ca45e394c72801a3b317df426c2" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42611 } } peers { permanent_uuid: "206b092bd0c94ae6b03a6428cf272ed9" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44197 } } peers { permanent_uuid: "a3e540d6e7584fda8d56a1f8af38b25d" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36099 } }
I20260806 00:58:02.885077 20539 leader_election.cc:288] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 206b092bd0c94ae6b03a6428cf272ed9 (127.27.77.193:44197), a3e540d6e7584fda8d56a1f8af38b25d (127.27.77.194:36099)
I20260806 00:58:02.889298 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.885369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:44197 (local address 127.27.77.195:39699)
0806 00:58:02.885531 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:02.885546 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:02.885674 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:02.886071 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:02.886074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:02.886104 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:02.886216 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.886222 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.887049 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.887052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.888016 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.888027 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.888798 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.888803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.888922 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.888943 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:58:02.889042 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 00:58:02.889132 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 00:58:02.889298 20523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.885579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:39699 (local address 127.27.77.193:44197)
0806 00:58:02.885788 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:02.885797 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:02.885816 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:58:02.885879 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:02.885883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:02.885939 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:02.886037 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:02.886348 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.886907 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.888157 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.888649 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.888804 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.888865 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:02.889069 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 00:58:02.889157 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 00:58:02.889482 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.885682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:36099 (local address 127.27.77.195:47177)
0806 00:58:02.886096 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:02.886109 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:02.886211 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:02.886508 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:02.886511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:02.886561 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:02.886686 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.886692 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.887466 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.887471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.888356 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:02.888366 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.889011 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.889015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:02.889132 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.889154 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:58:02.889255 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 00:58:02.889327 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":13,"thread_start_us":121,"threads_started":1}
I20260806 00:58:02.889467 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.885848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:47177 (local address 127.27.77.194:36099)
0806 00:58:02.885986 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:02.885991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:02.886187 (+ 196us) server_negotiation.cc:407] Connection header received
0806 00:58:02.886341 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:02.886349 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:02.886394 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:02.886489 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:02.886810 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.887340 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.888495 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.888883 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.889016 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.889080 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:02.889274 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 00:58:02.889352 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:58:02.890189 20046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36f12497b3d94a54bbc31de9c3f97b75" candidate_uuid: "33ba1ca45e394c72801a3b317df426c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "206b092bd0c94ae6b03a6428cf272ed9" is_pre_election: true
I20260806 00:58:02.890203 20238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36f12497b3d94a54bbc31de9c3f97b75" candidate_uuid: "33ba1ca45e394c72801a3b317df426c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3e540d6e7584fda8d56a1f8af38b25d" is_pre_election: true
I20260806 00:58:02.890550 20046 raft_consensus.cc:2454] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33ba1ca45e394c72801a3b317df426c2 in term 0.
I20260806 00:58:02.890615 20238 raft_consensus.cc:2454] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33ba1ca45e394c72801a3b317df426c2 in term 0.
I20260806 00:58:02.891165 20355 leader_election.cc:302] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 206b092bd0c94ae6b03a6428cf272ed9, 33ba1ca45e394c72801a3b317df426c2; no voters:
I20260806 00:58:02.891491 20539 raft_consensus.cc:2781] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:58:02.891659 20539 raft_consensus.cc:482] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:58:02.891724 20539 raft_consensus.cc:3037] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [term 0 FOLLOWER]: Advancing to term 1
W20260806 00:58:02.892159 20506 tablet.cc:2369] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:58:02.893049 20539 raft_consensus.cc:504] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ba1ca45e394c72801a3b317df426c2" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42611 } } peers { permanent_uuid: "206b092bd0c94ae6b03a6428cf272ed9" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44197 } } peers { permanent_uuid: "a3e540d6e7584fda8d56a1f8af38b25d" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36099 } }
I20260806 00:58:02.893472 20539 leader_election.cc:288] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [CANDIDATE]: Term 1 election: Requested vote from peers 206b092bd0c94ae6b03a6428cf272ed9 (127.27.77.193:44197), a3e540d6e7584fda8d56a1f8af38b25d (127.27.77.194:36099)
I20260806 00:58:02.894032 20046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36f12497b3d94a54bbc31de9c3f97b75" candidate_uuid: "33ba1ca45e394c72801a3b317df426c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "206b092bd0c94ae6b03a6428cf272ed9"
I20260806 00:58:02.894129 20238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36f12497b3d94a54bbc31de9c3f97b75" candidate_uuid: "33ba1ca45e394c72801a3b317df426c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3e540d6e7584fda8d56a1f8af38b25d"
I20260806 00:58:02.894217 20046 raft_consensus.cc:3037] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:02.894312 20238 raft_consensus.cc:3037] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:02.895614 20046 raft_consensus.cc:2454] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33ba1ca45e394c72801a3b317df426c2 in term 1.
I20260806 00:58:02.895634 20238 raft_consensus.cc:2454] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33ba1ca45e394c72801a3b317df426c2 in term 1.
I20260806 00:58:02.896044 20355 leader_election.cc:302] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 206b092bd0c94ae6b03a6428cf272ed9, 33ba1ca45e394c72801a3b317df426c2; no voters:
W20260806 00:58:02.896102 20320 tablet.cc:2369] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:58:02.896308 20539 raft_consensus.cc:2781] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:02.896584 20539 raft_consensus.cc:686] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [term 1 LEADER]: Becoming Leader. State: Replica: 33ba1ca45e394c72801a3b317df426c2, State: Running, Role: LEADER
I20260806 00:58:02.896955 20539 consensus_queue.cc:237] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ba1ca45e394c72801a3b317df426c2" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42611 } } peers { permanent_uuid: "206b092bd0c94ae6b03a6428cf272ed9" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44197 } } peers { permanent_uuid: "a3e540d6e7584fda8d56a1f8af38b25d" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36099 } }
I20260806 00:58:02.900679 19833 catalog_manager.cc:5495] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33ba1ca45e394c72801a3b317df426c2 (127.27.77.195). New cstate: current_term: 1 leader_uuid: "33ba1ca45e394c72801a3b317df426c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ba1ca45e394c72801a3b317df426c2" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 42611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "206b092bd0c94ae6b03a6428cf272ed9" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3e540d6e7584fda8d56a1f8af38b25d" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36099 } health_report { overall_health: UNKNOWN } } }
W20260806 00:58:02.908432 20107 tablet.cc:2369] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:58:02.940127 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.935696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41098 (local address 127.27.77.195:42611)
0806 00:58:02.935868 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:02.935884 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:02.935902 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:58:02.936146 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:02.936149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:02.936215 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:02.936345 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:58:02.936654 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.937218 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.938317 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:02.938575 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:02.938716 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:02.939097 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:58:02.939586 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:58:02.939772 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:02.939883 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 00:58:02.939934 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:58:02.947271 20046 raft_consensus.cc:1261] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9 [term 1 FOLLOWER]: Refusing update from remote peer 33ba1ca45e394c72801a3b317df426c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:58:02.947271 20238 raft_consensus.cc:1261] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d [term 1 FOLLOWER]: Refusing update from remote peer 33ba1ca45e394c72801a3b317df426c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:58:02.948179 20546 consensus_queue.cc:1035] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3e540d6e7584fda8d56a1f8af38b25d" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 36099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:58:02.948480 20539 consensus_queue.cc:1035] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "206b092bd0c94ae6b03a6428cf272ed9" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 44197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:58:02.958909 20551 mvcc.cc:205] Tried to move back new op lower bound from 7315365408543379456 to 7315365408348831744. Current Snapshot: MvccSnapshot[applied={T|T < 7315365408543379456}]
I20260806 00:58:02.967216 19833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:58:02.967391 19833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:58:02.970422 19833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e318263622e4087afdfb82b835852d3: Sending DeleteTablet for 3 replicas of tablet 36f12497b3d94a54bbc31de9c3f97b75
I20260806 00:58:02.971649 20413 tablet_service.cc:1514] Processing DeleteTablet for tablet 36f12497b3d94a54bbc31de9c3f97b75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:58:02 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:41096
I20260806 00:58:02.971709 20208 tablet_service.cc:1514] Processing DeleteTablet for tablet 36f12497b3d94a54bbc31de9c3f97b75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:58:02 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:55770
I20260806 00:58:02.972316 20564 tablet_replica.cc:330] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2: stopping tablet replica
I20260806 00:58:02.972323 20008 tablet_service.cc:1514] Processing DeleteTablet for tablet 36f12497b3d94a54bbc31de9c3f97b75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:58:02 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:37592
I20260806 00:58:02.972728 20564 raft_consensus.cc:2229] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:58:02.972838 20566 tablet_replica.cc:330] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9: stopping tablet replica
I20260806 00:58:02.973271 20564 raft_consensus.cc:2258] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:58:02.973238 20566 raft_consensus.cc:2229] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:58:02.973644 20566 raft_consensus.cc:2258] T 36f12497b3d94a54bbc31de9c3f97b75 P 206b092bd0c94ae6b03a6428cf272ed9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:58:02.973712 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 19924
I20260806 00:58:02.973803 20565 tablet_replica.cc:330] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d: stopping tablet replica
I20260806 00:58:02.974115 20565 raft_consensus.cc:2229] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:58:02.974447 20565 raft_consensus.cc:2258] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:58:02.976207 20565 ts_tablet_manager.cc:1905] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:58:02.979259 20565 ts_tablet_manager.cc:1918] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:58:02.979362 20565 log.cc:1196] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/36f12497b3d94a54bbc31de9c3f97b75
I20260806 00:58:02.979740 20565 ts_tablet_manager.cc:1939] T 36f12497b3d94a54bbc31de9c3f97b75 P a3e540d6e7584fda8d56a1f8af38b25d: Deleting consensus metadata
I20260806 00:58:02.981207 19816 catalog_manager.cc:4841] TS a3e540d6e7584fda8d56a1f8af38b25d (127.27.77.194:36099): tablet 36f12497b3d94a54bbc31de9c3f97b75 (table test-table [id=7f51682ad54d47b7a50124374f27733d]) successfully deleted
I20260806 00:58:02.984423 20564 ts_tablet_manager.cc:1905] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:58:02.987138 20564 ts_tablet_manager.cc:1918] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:58:02.987296 20564 log.cc:1196] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/36f12497b3d94a54bbc31de9c3f97b75
I20260806 00:58:02.987731 20564 ts_tablet_manager.cc:1939] T 36f12497b3d94a54bbc31de9c3f97b75 P 33ba1ca45e394c72801a3b317df426c2: Deleting consensus metadata
I20260806 00:58:02.988729 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 20111
W20260806 00:58:02.988775 19816 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:44197 (error 108)
I20260806 00:58:02.992017 19816 catalog_manager.cc:4841] TS 33ba1ca45e394c72801a3b317df426c2 (127.27.77.195:42611): tablet 36f12497b3d94a54bbc31de9c3f97b75 (table test-table [id=7f51682ad54d47b7a50124374f27733d]) successfully deleted
I20260806 00:58:02.996254 20526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:02.995926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:44197 (local address 127.0.0.1:37600)
0806 00:58:02.996060 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:02.996165 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:44197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 00:58:02.996495 19816 catalog_manager.cc:4584] TS 206b092bd0c94ae6b03a6428cf272ed9 (127.27.77.193:44197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 36f12497b3d94a54bbc31de9c3f97b75: Network error: Client connection negotiation failed: client connection to 127.27.77.193:44197: connect: Connection refused (error 111)
I20260806 00:58:03.005888 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 20327
I20260806 00:58:03.021117 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 19792
2026-08-06T00:58:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2743 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T00:58:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:03Z Disabled control of system clock
I20260806 00:58:03.060809 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:36733
--webserver_interface=127.27.77.254
--webserver_port=0
--builtin_ntp_servers=127.27.77.212:42043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:36733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:58:03.175480 20574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:03.175771 20574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:03.175808 20574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:03.179764 20574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:03.179823 20574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:03.179838 20574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:03.179855 20574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:03.179870 20574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:03.184628 20574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:42043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:36733
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:36733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:03.185689 20574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:03.187031 20574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:03.195873 20583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:03.195844 20582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:03.195968 20574 server_base.cc:1034] running on GCE node
W20260806 00:58:03.195866 20586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:03.196743 20574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:03.197887 20574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:03.199115 20574 hybrid_clock.cc:648] HybridClock initialized: now 1785977883199084 us; error 41 us; skew 500 ppm
I20260806 00:58:03.201232 20574 webserver.cc:460] Webserver started at http://127.27.77.254:34531/ using document root <none> and password file <none>
I20260806 00:58:03.201861 20574 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:03.201958 20574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:03.202198 20574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:03.204391 20574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "4aa0c32c73784ac6b9e096a97fd15d51"
format_stamp: "Formatted at 2026-08-06 00:58:03 on dist-test-slave-sb2z"
server_key: "82467056e46d74de9a396c32f55123d3"
server_key_iv: "101ec677b9b4ec9f6357d71eff5c9d54"
server_key_version: "encryptionkey@0"
I20260806 00:58:03.204991 20574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "4aa0c32c73784ac6b9e096a97fd15d51"
format_stamp: "Formatted at 2026-08-06 00:58:03 on dist-test-slave-sb2z"
server_key: "82467056e46d74de9a396c32f55123d3"
server_key_iv: "101ec677b9b4ec9f6357d71eff5c9d54"
server_key_version: "encryptionkey@0"
I20260806 00:58:03.208912 20574 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:58:03.211856 20594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:03.213272 20574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:58:03.213449 20574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "4aa0c32c73784ac6b9e096a97fd15d51"
format_stamp: "Formatted at 2026-08-06 00:58:03 on dist-test-slave-sb2z"
server_key: "82467056e46d74de9a396c32f55123d3"
server_key_iv: "101ec677b9b4ec9f6357d71eff5c9d54"
server_key_version: "encryptionkey@0"
I20260806 00:58:03.213603 20574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:03.226763 20574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:03.227487 20574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:03.227739 20574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:03.235301 20574 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:36733
I20260806 00:58:03.235325 20666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:36733 every 8 connection(s)
I20260806 00:58:03.236443 20574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:03.237536 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 20574
I20260806 00:58:03.237720 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:03.238180 27959 external_mini_cluster.cc:1428] Setting key a86c5a7cce475ef4b0134618df7b09f9
I20260806 00:58:03.240216 20668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:03.246871 20668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51: Bootstrap starting.
I20260806 00:58:03.248607 20673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:03.239746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42654 (local address 127.27.77.254:36733)
0806 00:58:03.240508 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:03.240526 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:03.240567 (+ 41us) server_negotiation.cc:407] Connection header received
0806 00:58:03.241375 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:03.241396 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:03.241824 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:03.242130 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:03.242502 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.243658 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.244866 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.245374 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.245520 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:03.246566 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:03.246571 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:03.246584 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:03.247054 (+ 470us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:03.247140 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:03.247164 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:03.247254 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:03.247460 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 00:58:03.247821 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":195,"threads_started":1}
I20260806 00:58:03.249476 20668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:03.250334 20668 log.cc:824] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:03.252552 20668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51: No bootstrap required, opened a new log
I20260806 00:58:03.256222 20668 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa0c32c73784ac6b9e096a97fd15d51" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 36733 } }
I20260806 00:58:03.256446 20668 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:03.256481 20668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa0c32c73784ac6b9e096a97fd15d51, State: Initialized, Role: FOLLOWER
I20260806 00:58:03.256953 20668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa0c32c73784ac6b9e096a97fd15d51" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 36733 } }
I20260806 00:58:03.257088 20668 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:03.257130 20668 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:03.257244 20668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:03.258159 20668 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa0c32c73784ac6b9e096a97fd15d51" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 36733 } }
I20260806 00:58:03.258499 20668 leader_election.cc:302] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4aa0c32c73784ac6b9e096a97fd15d51; no voters:
I20260806 00:58:03.258860 20668 leader_election.cc:288] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:03.259032 20675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:03.259323 20675 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [term 1 LEADER]: Becoming Leader. State: Replica: 4aa0c32c73784ac6b9e096a97fd15d51, State: Running, Role: LEADER
I20260806 00:58:03.259727 20675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa0c32c73784ac6b9e096a97fd15d51" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 36733 } }
I20260806 00:58:03.260169 20668 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:03.260908 20675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4aa0c32c73784ac6b9e096a97fd15d51. Latest consensus state: current_term: 1 leader_uuid: "4aa0c32c73784ac6b9e096a97fd15d51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa0c32c73784ac6b9e096a97fd15d51" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 36733 } } }
I20260806 00:58:03.261157 20675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:03.261404 20679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4aa0c32c73784ac6b9e096a97fd15d51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa0c32c73784ac6b9e096a97fd15d51" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 36733 } } }
I20260806 00:58:03.261595 20679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:03.261639 20690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:03.265230 20690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:03.271373 20690 catalog_manager.cc:1347] Generated new cluster ID: 0727108055514f48b9b7869b47170865
I20260806 00:58:03.271481 20690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:03.289502 20690 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:03.290710 20690 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:03.295243 20690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51: Generated new TSK 0
I20260806 00:58:03.296150 20690 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:03.302368 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:36733
--builtin_ntp_servers=127.27.77.212:42043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:58:03.428406 20708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:03.428746 20708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:03.428833 20708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:03.433079 20708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:03.433175 20708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:03.433275 20708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:58:03.438248 20708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:42043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:36733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:03.439601 20708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:03.441190 20708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:03.453299 20720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:03.453310 20714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:03.453698 20708 server_base.cc:1034] running on GCE node
W20260806 00:58:03.453310 20713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:03.454378 20708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:03.455049 20708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:03.456239 20708 hybrid_clock.cc:648] HybridClock initialized: now 1785977883456220 us; error 32 us; skew 500 ppm
I20260806 00:58:03.460422 20708 webserver.cc:460] Webserver started at http://127.27.77.193:35687/ using document root <none> and password file <none>
I20260806 00:58:03.461166 20708 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:03.461266 20708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:03.461617 20708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:03.464004 20708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "2207a21c7d164992a169000b72407dae"
format_stamp: "Formatted at 2026-08-06 00:58:03 on dist-test-slave-sb2z"
server_key: "d9bfde98c0d577cf6749a38da4885517"
server_key_iv: "6447b59a3af0e25b5f17412247b90744"
server_key_version: "encryptionkey@0"
I20260806 00:58:03.464638 20708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "2207a21c7d164992a169000b72407dae"
format_stamp: "Formatted at 2026-08-06 00:58:03 on dist-test-slave-sb2z"
server_key: "d9bfde98c0d577cf6749a38da4885517"
server_key_iv: "6447b59a3af0e25b5f17412247b90744"
server_key_version: "encryptionkey@0"
I20260806 00:58:03.468756 20708 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:58:03.471424 20727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:03.472792 20708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:58:03.472944 20708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "2207a21c7d164992a169000b72407dae"
format_stamp: "Formatted at 2026-08-06 00:58:03 on dist-test-slave-sb2z"
server_key: "d9bfde98c0d577cf6749a38da4885517"
server_key_iv: "6447b59a3af0e25b5f17412247b90744"
server_key_version: "encryptionkey@0"
I20260806 00:58:03.473078 20708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:03.495461 20708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:03.496372 20708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:03.496622 20708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:03.497282 20708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:58:03.498323 20708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:58:03.498409 20708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:03.498507 20708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:58:03.498569 20708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:03.508926 20708 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:40201
I20260806 00:58:03.508988 20888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:40201 every 8 connection(s)
I20260806 00:58:03.510005 20708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:58:03.511791 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 20708
I20260806 00:58:03.511943 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:58:03.512333 27959 external_mini_cluster.cc:1428] Setting key f395f4b2eaff5de54d6389a78ea27f3d
I20260806 00:58:03.514726 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:36733
--builtin_ntp_servers=127.27.77.212:42043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:58:03.521051 20673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:03.512248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:38445 (local address 127.27.77.254:36733)
0806 00:58:03.512415 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:03.512420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:03.513782 (+ 1362us) server_negotiation.cc:407] Connection header received
0806 00:58:03.515437 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:03.515443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:03.515520 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:03.515655 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:03.516811 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.517436 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.518767 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.519001 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.519123 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:03.520466 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:03.520470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:03.520472 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:03.520485 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:03.520505 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:03.520514 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:03.520603 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:03.520780 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 00:58:03.520872 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 00:58:03.521994 20891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:03.512712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:36733 (local address 127.27.77.193:38445)
0806 00:58:03.513568 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:03.513628 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:03.515200 (+ 1572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:03.515871 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:03.515878 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:03.515980 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:03.516646 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:03.516665 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.517633 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.517636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:03.518632 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:03.518643 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.519174 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.519178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:03.519353 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:03.520165 (+ 812us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:58:03.520167 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:03.520211 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:58:03.520286 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:03.520619 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:03.520631 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:03.520643 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 00:58:03.520777 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 00:58:03.521128 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":173,"threads_started":1}
I20260806 00:58:03.523663 20889 heartbeater.cc:347] Connected to a master server at 127.27.77.254:36733
I20260806 00:58:03.523943 20889 heartbeater.cc:464] Registering TS with master...
I20260806 00:58:03.524538 20889 heartbeater.cc:511] Master 127.27.77.254:36733 requested a full tablet report, sending...
I20260806 00:58:03.526401 20620 ts_manager.cc:194] Registered new tserver with Master: 2207a21c7d164992a169000b72407dae (127.27.77.193:40201)
I20260806 00:58:03.528237 20620 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.77.193:38445
W20260806 00:58:03.635630 20893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:03.635916 20893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:03.635957 20893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:03.640018 20893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:03.640082 20893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:03.640138 20893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:58:03.645193 20893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:42043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:36733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:03.646309 20893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:03.647895 20893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:03.658887 20898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:03.658892 20899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:03.658972 20901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:03.658972 20893 server_base.cc:1034] running on GCE node
I20260806 00:58:03.659785 20893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:03.660646 20893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:03.661864 20893 hybrid_clock.cc:648] HybridClock initialized: now 1785977883661845 us; error 35 us; skew 500 ppm
I20260806 00:58:03.664220 20893 webserver.cc:460] Webserver started at http://127.27.77.194:39747/ using document root <none> and password file <none>
I20260806 00:58:03.665056 20893 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:03.665161 20893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:03.665439 20893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:03.668519 20893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "eb23d833bbc1477c883c192776440449"
format_stamp: "Formatted at 2026-08-06 00:58:03 on dist-test-slave-sb2z"
server_key: "bc1c9a20a2d88dba35c1ff9f68f5bdf9"
server_key_iv: "839851f9e245bb35cc90e27cec8657cc"
server_key_version: "encryptionkey@0"
I20260806 00:58:03.669270 20893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "eb23d833bbc1477c883c192776440449"
format_stamp: "Formatted at 2026-08-06 00:58:03 on dist-test-slave-sb2z"
server_key: "bc1c9a20a2d88dba35c1ff9f68f5bdf9"
server_key_iv: "839851f9e245bb35cc90e27cec8657cc"
server_key_version: "encryptionkey@0"
I20260806 00:58:03.673590 20893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 00:58:03.676507 20910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:03.678107 20893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:03.678262 20893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "eb23d833bbc1477c883c192776440449"
format_stamp: "Formatted at 2026-08-06 00:58:03 on dist-test-slave-sb2z"
server_key: "bc1c9a20a2d88dba35c1ff9f68f5bdf9"
server_key_iv: "839851f9e245bb35cc90e27cec8657cc"
server_key_version: "encryptionkey@0"
I20260806 00:58:03.678395 20893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:03.704171 20893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:03.705030 20893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:03.705279 20893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:03.706004 20893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:58:03.707152 20893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:58:03.707239 20893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:03.707324 20893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:58:03.707376 20893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:03.718380 20893 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:41633
I20260806 00:58:03.718403 21060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:41633 every 8 connection(s)
I20260806 00:58:03.719475 20893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:58:03.723635 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 20893
I20260806 00:58:03.723793 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:58:03.724092 27959 external_mini_cluster.cc:1428] Setting key 9636b00a88f2a7901febd5b542df97d3
I20260806 00:58:03.726078 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:36733
--builtin_ntp_servers=127.27.77.212:42043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:58:03.730043 20673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:03.721685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:54499 (local address 127.27.77.254:36733)
0806 00:58:03.721866 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:03.721870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:03.722982 (+ 1112us) server_negotiation.cc:407] Connection header received
0806 00:58:03.724667 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:03.724670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:03.724734 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:03.724848 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:03.725799 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.726425 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.727631 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.727988 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.728119 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:03.729354 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:03.729361 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:03.729366 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:03.729385 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:03.729414 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:03.729437 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:03.729521 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:03.729744 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 00:58:03.729828 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 00:58:03.731055 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:03.722102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:36733 (local address 127.27.77.194:54499)
0806 00:58:03.722839 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:03.722883 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:03.724427 (+ 1544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:03.725040 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:03.725047 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:03.725135 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:03.725646 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:03.725659 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.726577 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.726579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:03.727478 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:03.727488 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.728185 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.728188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:03.728347 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:03.729051 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:58:03.729053 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:03.729102 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:58:03.729174 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:03.729557 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:03.729567 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:03.729582 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 00:58:03.729708 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 00:58:03.730114 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":130,"threads_started":1}
I20260806 00:58:03.732425 21063 heartbeater.cc:347] Connected to a master server at 127.27.77.254:36733
I20260806 00:58:03.732745 21063 heartbeater.cc:464] Registering TS with master...
I20260806 00:58:03.733505 21063 heartbeater.cc:511] Master 127.27.77.254:36733 requested a full tablet report, sending...
I20260806 00:58:03.734812 20620 ts_manager.cc:194] Registered new tserver with Master: eb23d833bbc1477c883c192776440449 (127.27.77.194:41633)
I20260806 00:58:03.735433 20620 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.77.194:54499
W20260806 00:58:03.850189 21068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:03.850484 21068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:03.850528 21068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:03.855130 21068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:03.855196 21068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:03.855283 21068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:58:03.860460 21068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:42043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:36733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:03.861877 21068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:03.863359 21068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:03.873660 21074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:03.873714 21073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:03.873660 21078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:03.874155 21068 server_base.cc:1034] running on GCE node
I20260806 00:58:03.874620 21068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:03.875319 21068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:03.876554 21068 hybrid_clock.cc:648] HybridClock initialized: now 1785977883876531 us; error 42 us; skew 500 ppm
I20260806 00:58:03.878861 21068 webserver.cc:460] Webserver started at http://127.27.77.195:46563/ using document root <none> and password file <none>
I20260806 00:58:03.879606 21068 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:03.879709 21068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:03.879969 21068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:03.882391 21068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "44cd595702114b66a23849d6a844882a"
format_stamp: "Formatted at 2026-08-06 00:58:03 on dist-test-slave-sb2z"
server_key: "b872b6ce5be9aa2987d090231b852728"
server_key_iv: "b24009e4386cb5b0ea3ce78a3944ec66"
server_key_version: "encryptionkey@0"
I20260806 00:58:03.883033 21068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "44cd595702114b66a23849d6a844882a"
format_stamp: "Formatted at 2026-08-06 00:58:03 on dist-test-slave-sb2z"
server_key: "b872b6ce5be9aa2987d090231b852728"
server_key_iv: "b24009e4386cb5b0ea3ce78a3944ec66"
server_key_version: "encryptionkey@0"
I20260806 00:58:03.887100 21068 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:58:03.890024 21084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:03.891456 21068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:58:03.891665 21068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "44cd595702114b66a23849d6a844882a"
format_stamp: "Formatted at 2026-08-06 00:58:03 on dist-test-slave-sb2z"
server_key: "b872b6ce5be9aa2987d090231b852728"
server_key_iv: "b24009e4386cb5b0ea3ce78a3944ec66"
server_key_version: "encryptionkey@0"
I20260806 00:58:03.891811 21068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:03.904340 21068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:03.905153 21068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:03.905385 21068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:03.906060 21068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:58:03.907109 21068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:58:03.907200 21068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:03.907300 21068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:58:03.907354 21068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:03.917368 21068 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:39769
I20260806 00:58:03.917416 21250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:39769 every 8 connection(s)
I20260806 00:58:03.918423 21068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:58:03.925009 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 21068
I20260806 00:58:03.925155 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:58:03.925464 27959 external_mini_cluster.cc:1428] Setting key 92589ce471c38003adfaba0931af0d02
I20260806 00:58:03.929360 20673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:03.920577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:40479 (local address 127.27.77.254:36733)
0806 00:58:03.920786 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:03.920791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:03.922007 (+ 1216us) server_negotiation.cc:407] Connection header received
0806 00:58:03.923436 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:03.923439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:03.923497 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:03.923654 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:03.924688 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.925365 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.927148 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.927385 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.927505 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:03.928816 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:03.928820 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:03.928824 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:03.928842 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:03.928863 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:03.928871 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:03.928953 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:03.929136 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 00:58:03.929205 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260806 00:58:03.930362 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:03.921064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:36733 (local address 127.27.77.195:40479)
0806 00:58:03.921790 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:03.921835 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:03.923185 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:03.923838 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:03.923845 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:03.923935 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:03.924499 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:03.924520 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.925810 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.925816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:03.927005 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:03.927017 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.927591 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.927597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:03.927772 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:03.928507 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:58:03.928508 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:03.928560 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:58:03.928634 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:03.928958 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:03.928970 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:03.928976 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 00:58:03.929136 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 00:58:03.929460 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":129,"threads_started":1}
I20260806 00:58:03.931659 21252 heartbeater.cc:347] Connected to a master server at 127.27.77.254:36733
I20260806 00:58:03.931922 21252 heartbeater.cc:464] Registering TS with master...
I20260806 00:58:03.932520 21252 heartbeater.cc:511] Master 127.27.77.254:36733 requested a full tablet report, sending...
I20260806 00:58:03.933708 20620 ts_manager.cc:194] Registered new tserver with Master: 44cd595702114b66a23849d6a844882a (127.27.77.195:39769)
I20260806 00:58:03.934288 20620 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.77.195:40479
I20260806 00:58:03.940186 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:58:03.946640 20673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:03.942527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42670 (local address 127.27.77.254:36733)
0806 00:58:03.942668 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:03.942672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:03.942839 (+ 167us) server_negotiation.cc:407] Connection header received
0806 00:58:03.942950 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:03.942952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:03.942995 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:03.943089 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:03.943346 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.943945 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.944935 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.945156 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.945247 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:03.946156 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:03.946159 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:03.946161 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:03.946173 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:03.946195 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:03.946210 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:03.946335 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:03.946472 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 00:58:03.946509 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:58:03.950048 20620 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:58:03.953313 20620 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:58:03.969830 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:03.963129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:39769 (local address 127.0.0.1:34350)
0806 00:58:03.963557 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:03.963617 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:03.963752 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:03.965367 (+ 1615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:03.965370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:03.965390 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:03.965662 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:03.965672 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.966690 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.966696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:03.968085 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:03.968097 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.969209 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.969213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:03.969337 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:03.969385 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 00:58:03.969497 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 00:58:03.969656 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":126,"thread_start_us":119,"threads_started":1}
I20260806 00:58:03.970065 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:03.964084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49544 (local address 127.27.77.194:41633)
0806 00:58:03.964548 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:03.964569 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:03.964591 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:58:03.964694 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:03.964700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:03.964927 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:03.965114 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:03.965835 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.967030 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.968632 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.969402 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.969521 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:03.969708 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:03.969761 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 00:58:03.969869 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":129,"threads_started":1}
I20260806 00:58:03.970156 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:03.964215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47922 (local address 127.27.77.193:40201)
0806 00:58:03.964841 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:03.964854 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:03.964879 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:58:03.964963 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:03.964967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:03.965141 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:03.965300 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:03.965821 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.967049 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.968554 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.969203 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.969637 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:03.969817 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:03.969872 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 00:58:03.969982 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":119,"threads_started":1}
I20260806 00:58:03.970451 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:03.963984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:40201 (local address 127.0.0.1:47922)
0806 00:58:03.964473 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:03.964489 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:03.964585 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:03.965316 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:03.965323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:03.965362 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:03.965664 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:03.965674 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.967189 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.967192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:03.968419 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:03.968429 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.969337 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.969340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:03.969412 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:03.969430 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 00:58:03.969476 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 00:58:03.970298 (+ 822us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":808,"spinlock_wait_cycles":18816,"thread_start_us":106,"threads_started":1}
I20260806 00:58:03.970495 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:03.963477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:41633 (local address 127.0.0.1:49544)
0806 00:58:03.964316 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:03.964331 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:03.964426 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:03.965316 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:03.965323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:03.965361 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:03.965696 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:03.965706 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.967168 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.967172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:03.968493 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:03.968503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.969524 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.969528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:03.969614 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:03.969637 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:58:03.969700 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 00:58:03.969772 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"mutex_wait_us":168,"spinlock_wait_cycles":15104,"thread_start_us":139,"threads_started":1}
I20260806 00:58:03.971096 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:03.963357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34350 (local address 127.27.77.195:39769)
0806 00:58:03.964841 (+ 1484us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:03.964854 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:03.964879 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:58:03.964956 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:03.964961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:03.965142 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:03.965610 (+ 468us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:03.965766 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.966562 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.968234 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:03.969066 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:03.969182 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:03.969396 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:03.970775 (+ 1379us) server_negotiation.cc:299] Negotiation successful
0806 00:58:03.970894 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1377,"thread_start_us":211,"threads_started":1}
I20260806 00:58:03.973575 21160 tablet_service.cc:1467] Processing CreateTablet for tablet 0b18c7c317684c4bb7fd1d9d286bce7f (DEFAULT_TABLE table=test-table [id=c84c908b6b6148d684c2bcdd07cdecfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:58:03.973493 20973 tablet_service.cc:1467] Processing CreateTablet for tablet 0b18c7c317684c4bb7fd1d9d286bce7f (DEFAULT_TABLE table=test-table [id=c84c908b6b6148d684c2bcdd07cdecfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:58:03.974570 21160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b18c7c317684c4bb7fd1d9d286bce7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:03.974570 20973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b18c7c317684c4bb7fd1d9d286bce7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:03.981659 21278 tablet_bootstrap.cc:492] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a: Bootstrap starting.
I20260806 00:58:03.982200 20792 tablet_service.cc:1467] Processing CreateTablet for tablet 0b18c7c317684c4bb7fd1d9d286bce7f (DEFAULT_TABLE table=test-table [id=c84c908b6b6148d684c2bcdd07cdecfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:58:03.983608 20792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b18c7c317684c4bb7fd1d9d286bce7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:03.983933 21279 tablet_bootstrap.cc:492] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449: Bootstrap starting.
I20260806 00:58:03.984987 21278 tablet_bootstrap.cc:654] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:03.985698 21278 log.cc:824] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:03.986244 21279 tablet_bootstrap.cc:654] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:03.987152 21279 log.cc:824] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:03.988077 21278 tablet_bootstrap.cc:492] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a: No bootstrap required, opened a new log
I20260806 00:58:03.988322 21278 ts_tablet_manager.cc:1397] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:58:03.989404 21279 tablet_bootstrap.cc:492] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449: No bootstrap required, opened a new log
I20260806 00:58:03.989641 21279 ts_tablet_manager.cc:1397] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 00:58:03.992434 21283 tablet_bootstrap.cc:492] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae: Bootstrap starting.
I20260806 00:58:03.992398 21278 raft_consensus.cc:348] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cd595702114b66a23849d6a844882a" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39769 } } peers { permanent_uuid: "2207a21c7d164992a169000b72407dae" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40201 } } peers { permanent_uuid: "eb23d833bbc1477c883c192776440449" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 41633 } }
I20260806 00:58:03.992868 21278 raft_consensus.cc:374] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:03.992991 21278 raft_consensus.cc:729] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44cd595702114b66a23849d6a844882a, State: Initialized, Role: FOLLOWER
I20260806 00:58:03.993928 21278 consensus_queue.cc:260] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cd595702114b66a23849d6a844882a" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39769 } } peers { permanent_uuid: "2207a21c7d164992a169000b72407dae" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40201 } } peers { permanent_uuid: "eb23d833bbc1477c883c192776440449" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 41633 } }
I20260806 00:58:03.994234 21279 raft_consensus.cc:348] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cd595702114b66a23849d6a844882a" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39769 } } peers { permanent_uuid: "2207a21c7d164992a169000b72407dae" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40201 } } peers { permanent_uuid: "eb23d833bbc1477c883c192776440449" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 41633 } }
I20260806 00:58:03.994608 21279 raft_consensus.cc:374] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:03.994709 21279 raft_consensus.cc:729] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb23d833bbc1477c883c192776440449, State: Initialized, Role: FOLLOWER
I20260806 00:58:03.994913 21252 heartbeater.cc:503] Master 127.27.77.254:36733 was elected leader, sending a full tablet report...
I20260806 00:58:03.995287 21278 ts_tablet_manager.cc:1428] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 00:58:03.995760 21283 tablet_bootstrap.cc:654] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:03.996102 21279 consensus_queue.cc:260] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cd595702114b66a23849d6a844882a" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39769 } } peers { permanent_uuid: "2207a21c7d164992a169000b72407dae" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40201 } } peers { permanent_uuid: "eb23d833bbc1477c883c192776440449" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 41633 } }
I20260806 00:58:03.996939 21063 heartbeater.cc:503] Master 127.27.77.254:36733 was elected leader, sending a full tablet report...
I20260806 00:58:03.996932 21283 log.cc:824] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:03.997177 21279 ts_tablet_manager.cc:1428] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449: Time spent starting tablet: real 0.007s user 0.006s sys 0.002s
I20260806 00:58:03.999444 21283 tablet_bootstrap.cc:492] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae: No bootstrap required, opened a new log
I20260806 00:58:03.999711 21283 ts_tablet_manager.cc:1397] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260806 00:58:04.003757 21283 raft_consensus.cc:348] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cd595702114b66a23849d6a844882a" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39769 } } peers { permanent_uuid: "2207a21c7d164992a169000b72407dae" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40201 } } peers { permanent_uuid: "eb23d833bbc1477c883c192776440449" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 41633 } }
I20260806 00:58:04.004045 21283 raft_consensus.cc:374] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:04.004132 21283 raft_consensus.cc:729] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2207a21c7d164992a169000b72407dae, State: Initialized, Role: FOLLOWER
I20260806 00:58:04.004596 21283 consensus_queue.cc:260] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cd595702114b66a23849d6a844882a" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39769 } } peers { permanent_uuid: "2207a21c7d164992a169000b72407dae" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40201 } } peers { permanent_uuid: "eb23d833bbc1477c883c192776440449" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 41633 } }
I20260806 00:58:04.005400 20889 heartbeater.cc:503] Master 127.27.77.254:36733 was elected leader, sending a full tablet report...
I20260806 00:58:04.005683 21283 ts_tablet_manager.cc:1428] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260806 00:58:04.011389 20890 tablet.cc:2369] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:58:04.099261 21284 raft_consensus.cc:482] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:58:04.099575 21284 raft_consensus.cc:504] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cd595702114b66a23849d6a844882a" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39769 } } peers { permanent_uuid: "2207a21c7d164992a169000b72407dae" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40201 } } peers { permanent_uuid: "eb23d833bbc1477c883c192776440449" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 41633 } }
I20260806 00:58:04.100819 21284 leader_election.cc:288] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2207a21c7d164992a169000b72407dae (127.27.77.193:40201), eb23d833bbc1477c883c192776440449 (127.27.77.194:41633)
I20260806 00:58:04.106015 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:04.101259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:38095 (local address 127.27.77.193:40201)
0806 00:58:04.101402 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:04.101420 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:04.101762 (+ 342us) server_negotiation.cc:407] Connection header received
0806 00:58:04.101842 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:04.101845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:04.101888 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:04.101948 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:04.102256 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:04.102972 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:04.104662 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:04.105277 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:04.105422 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:04.105475 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:04.105789 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 00:58:04.105876 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:58:04.106015 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:04.101260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:37359 (local address 127.27.77.194:41633)
0806 00:58:04.101392 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:04.101396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:04.101413 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:58:04.101562 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:04.101570 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:04.101623 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:04.101688 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:04.102070 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:04.102847 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:04.104656 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:04.105307 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:04.105422 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:04.105475 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:04.105789 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 00:58:04.105876 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 00:58:04.106040 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:04.101130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:41633 (local address 127.27.77.195:37359)
0806 00:58:04.101272 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:04.101290 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:04.101402 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:04.101743 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:04.101746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:04.101760 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:04.101901 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:04.101908 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:04.102987 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:04.102990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:04.104471 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:04.104484 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:04.105441 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:04.105445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:04.105606 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:04.105665 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 00:58:04.105740 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 00:58:04.105844 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":92}
I20260806 00:58:04.106048 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:04.101136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:40201 (local address 127.27.77.195:38095)
0806 00:58:04.101622 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:04.101633 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:04.101757 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:04.102000 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:04.102004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:04.102026 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:04.102133 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:04.102137 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:04.103103 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:04.103106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:04.104471 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:04.104484 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:04.105441 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:04.105445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:04.105611 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:04.105655 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 00:58:04.105740 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 00:58:04.105844 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":238,"thread_start_us":109,"threads_started":1}
I20260806 00:58:04.106866 21000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b18c7c317684c4bb7fd1d9d286bce7f" candidate_uuid: "44cd595702114b66a23849d6a844882a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb23d833bbc1477c883c192776440449" is_pre_election: true
I20260806 00:58:04.106938 20820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b18c7c317684c4bb7fd1d9d286bce7f" candidate_uuid: "44cd595702114b66a23849d6a844882a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2207a21c7d164992a169000b72407dae" is_pre_election: true
I20260806 00:58:04.107245 21000 raft_consensus.cc:2454] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44cd595702114b66a23849d6a844882a in term 0.
I20260806 00:58:04.107347 20820 raft_consensus.cc:2454] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44cd595702114b66a23849d6a844882a in term 0.
I20260806 00:58:04.107985 21091 leader_election.cc:302] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2207a21c7d164992a169000b72407dae, 44cd595702114b66a23849d6a844882a, eb23d833bbc1477c883c192776440449; no voters:
I20260806 00:58:04.108356 21284 raft_consensus.cc:2781] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:58:04.108448 21284 raft_consensus.cc:482] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:58:04.108480 21284 raft_consensus.cc:3037] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:04.109799 21284 raft_consensus.cc:504] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cd595702114b66a23849d6a844882a" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39769 } } peers { permanent_uuid: "2207a21c7d164992a169000b72407dae" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40201 } } peers { permanent_uuid: "eb23d833bbc1477c883c192776440449" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 41633 } }
I20260806 00:58:04.110275 21284 leader_election.cc:288] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [CANDIDATE]: Term 1 election: Requested vote from peers 2207a21c7d164992a169000b72407dae (127.27.77.193:40201), eb23d833bbc1477c883c192776440449 (127.27.77.194:41633)
I20260806 00:58:04.110817 21000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b18c7c317684c4bb7fd1d9d286bce7f" candidate_uuid: "44cd595702114b66a23849d6a844882a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb23d833bbc1477c883c192776440449"
I20260806 00:58:04.110816 20820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b18c7c317684c4bb7fd1d9d286bce7f" candidate_uuid: "44cd595702114b66a23849d6a844882a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2207a21c7d164992a169000b72407dae"
I20260806 00:58:04.110971 20820 raft_consensus.cc:3037] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:04.110971 21000 raft_consensus.cc:3037] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:04.112484 20820 raft_consensus.cc:2454] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44cd595702114b66a23849d6a844882a in term 1.
I20260806 00:58:04.112484 21000 raft_consensus.cc:2454] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44cd595702114b66a23849d6a844882a in term 1.
I20260806 00:58:04.112923 21090 leader_election.cc:302] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2207a21c7d164992a169000b72407dae, 44cd595702114b66a23849d6a844882a, eb23d833bbc1477c883c192776440449; no voters:
I20260806 00:58:04.113169 21284 raft_consensus.cc:2781] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:04.113494 21284 raft_consensus.cc:686] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [term 1 LEADER]: Becoming Leader. State: Replica: 44cd595702114b66a23849d6a844882a, State: Running, Role: LEADER
I20260806 00:58:04.113906 21284 consensus_queue.cc:237] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cd595702114b66a23849d6a844882a" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39769 } } peers { permanent_uuid: "2207a21c7d164992a169000b72407dae" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40201 } } peers { permanent_uuid: "eb23d833bbc1477c883c192776440449" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 41633 } }
I20260806 00:58:04.117584 20620 catalog_manager.cc:5495] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a reported cstate change: term changed from 0 to 1, leader changed from <none> to 44cd595702114b66a23849d6a844882a (127.27.77.195). New cstate: current_term: 1 leader_uuid: "44cd595702114b66a23849d6a844882a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44cd595702114b66a23849d6a844882a" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 39769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2207a21c7d164992a169000b72407dae" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb23d833bbc1477c883c192776440449" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 41633 } health_report { overall_health: UNKNOWN } } }
I20260806 00:58:04.164721 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:04.160761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34366 (local address 127.27.77.195:39769)
0806 00:58:04.160907 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:04.160920 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:04.160936 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:58:04.161014 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:04.161016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:04.161074 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:04.161202 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:58:04.161537 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:04.162037 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:04.162968 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:04.163212 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:04.163296 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:04.163776 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:58:04.164220 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:58:04.164341 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:04.164530 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 00:58:04.164566 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 00:58:04.169478 21253 tablet.cc:2369] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:58:04.171228 21000 raft_consensus.cc:1261] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449 [term 1 FOLLOWER]: Refusing update from remote peer 44cd595702114b66a23849d6a844882a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:58:04.171245 20820 raft_consensus.cc:1261] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae [term 1 FOLLOWER]: Refusing update from remote peer 44cd595702114b66a23849d6a844882a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:58:04.172047 21284 consensus_queue.cc:1035] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb23d833bbc1477c883c192776440449" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:58:04.172273 21291 consensus_queue.cc:1035] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2207a21c7d164992a169000b72407dae" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 40201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:58:04.182502 21300 mvcc.cc:205] Tried to move back new op lower bound from 7315365413556883456 to 7315365413333680128. Current Snapshot: MvccSnapshot[applied={T|T < 7315365413556883456}]
I20260806 00:58:04.190398 20619 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:58:04.190598 20619 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:58:04.193277 20619 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4aa0c32c73784ac6b9e096a97fd15d51: Sending DeleteTablet for 3 replicas of tablet 0b18c7c317684c4bb7fd1d9d286bce7f
I20260806 00:58:04.194267 20973 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b18c7c317684c4bb7fd1d9d286bce7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:58:04 UTC) from {username='slave'} at 127.0.0.1:49544
I20260806 00:58:04.194243 20792 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b18c7c317684c4bb7fd1d9d286bce7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:58:04 UTC) from {username='slave'} at 127.0.0.1:47922
I20260806 00:58:04.194418 21160 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b18c7c317684c4bb7fd1d9d286bce7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:58:04 UTC) from {username='slave'} at 127.0.0.1:34350
I20260806 00:58:04.194790 21312 tablet_replica.cc:330] T 0b18c7c317684c4bb7fd1d9d286bce7f P 2207a21c7d164992a169000b72407dae: stopping tablet replica
I20260806 00:58:04.195053 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 20708
I20260806 00:58:04.202989 21315 tablet_replica.cc:330] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a: stopping tablet replica
I20260806 00:58:04.203559 21315 raft_consensus.cc:2229] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:58:04.204144 21315 raft_consensus.cc:2258] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:58:04.206555 21315 ts_tablet_manager.cc:1905] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:58:04.210120 21315 ts_tablet_manager.cc:1918] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:58:04.210315 21315 log.cc:1196] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/0b18c7c317684c4bb7fd1d9d286bce7f
I20260806 00:58:04.210716 21315 ts_tablet_manager.cc:1939] T 0b18c7c317684c4bb7fd1d9d286bce7f P 44cd595702114b66a23849d6a844882a: Deleting consensus metadata
I20260806 00:58:04.195073 21314 tablet_replica.cc:330] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449: stopping tablet replica
I20260806 00:58:04.214475 21314 raft_consensus.cc:2229] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:58:04.214949 21314 raft_consensus.cc:2258] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:58:04.218986 20602 catalog_manager.cc:4841] TS 44cd595702114b66a23849d6a844882a (127.27.77.195:39769): tablet 0b18c7c317684c4bb7fd1d9d286bce7f (table test-table [id=c84c908b6b6148d684c2bcdd07cdecfd]) successfully deleted
I20260806 00:58:04.220367 21314 ts_tablet_manager.cc:1905] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:58:04.223933 21314 ts_tablet_manager.cc:1918] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:58:04.224087 21314 log.cc:1196] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/0b18c7c317684c4bb7fd1d9d286bce7f
I20260806 00:58:04.224516 21314 ts_tablet_manager.cc:1939] T 0b18c7c317684c4bb7fd1d9d286bce7f P eb23d833bbc1477c883c192776440449: Deleting consensus metadata
W20260806 00:58:04.225291 20602 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:40201 (error 108)
I20260806 00:58:04.225437 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 20893
I20260806 00:58:04.232784 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:04.232413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:40201 (local address 127.0.0.1:47930)
0806 00:58:04.232567 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:04.232677 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:40201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 00:58:04.233168 20602 catalog_manager.cc:4584] TS 2207a21c7d164992a169000b72407dae (127.27.77.193:40201): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0b18c7c317684c4bb7fd1d9d286bce7f: Network error: Client connection negotiation failed: client connection to 127.27.77.193:40201: connect: Connection refused (error 111)
I20260806 00:58:04.243423 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:04.243122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:41633 (local address 127.0.0.1:49556)
0806 00:58:04.243265 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:04.243329 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.194:41633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 00:58:04.243669 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 21068
I20260806 00:58:04.261189 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 20574
2026-08-06T00:58:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1238 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:04 dist-test-slave-sb2z krb5kdc[21320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:04 dist-test-slave-sb2z krb5kdc[21320](info): set up 2 sockets
Aug 06 00:58:04 dist-test-slave-sb2z krb5kdc[21320](info): commencing operation
krb5kdc: starting...
W20260806 00:58:05.482604 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:05 dist-test-slave-sb2z krb5kdc[21320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:58:05.631997 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:42425
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:42425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:05.752651 21336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:05.753021 21336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:05.753121 21336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:05.757316 21336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:05.757433 21336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:05.757469 21336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:05.757511 21336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:05.757544 21336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:05.763221 21336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:42425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:42425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:05.764498 21336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:05.765607 21336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:05.774797 21346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:05.774766 21342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:05.774770 21344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:05.774995 21336 server_base.cc:1034] running on GCE node
I20260806 00:58:05.775844 21336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:05.777004 21336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:05.778208 21336 hybrid_clock.cc:648] HybridClock initialized: now 1785977885778191 us; error 30 us; skew 500 ppm
Aug 06 00:58:05 dist-test-slave-sb2z krb5kdc[21320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977885, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:05.781353 21336 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:58:05.782371 21336 webserver.cc:460] Webserver started at http://127.27.77.254:37463/ using document root <none> and password file <none>
I20260806 00:58:05.782982 21336 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:05.783092 21336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:05.783345 21336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:05.785553 21336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "548a2d8b54944254b05b20d09c21b99f"
format_stamp: "Formatted at 2026-08-06 00:58:05 on dist-test-slave-sb2z"
server_key: "f0cd8481f95ff0f7fe2c3664da8cccdb"
server_key_iv: "408004b337cf7511b9a77e1567818fa8"
server_key_version: "encryptionkey@0"
I20260806 00:58:05.786207 21336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "548a2d8b54944254b05b20d09c21b99f"
format_stamp: "Formatted at 2026-08-06 00:58:05 on dist-test-slave-sb2z"
server_key: "f0cd8481f95ff0f7fe2c3664da8cccdb"
server_key_iv: "408004b337cf7511b9a77e1567818fa8"
server_key_version: "encryptionkey@0"
I20260806 00:58:05.790359 21336 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 00:58:05.793064 21355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:05.794624 21336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 00:58:05.794801 21336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "548a2d8b54944254b05b20d09c21b99f"
format_stamp: "Formatted at 2026-08-06 00:58:05 on dist-test-slave-sb2z"
server_key: "f0cd8481f95ff0f7fe2c3664da8cccdb"
server_key_iv: "408004b337cf7511b9a77e1567818fa8"
server_key_version: "encryptionkey@0"
I20260806 00:58:05.794951 21336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:05.808211 21336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:05.811858 21336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:05.812098 21336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:05.819960 21336 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:42425
I20260806 00:58:05.819981 21426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:42425 every 8 connection(s)
I20260806 00:58:05.821033 21336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:05.824204 21428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:05.830761 21428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f: Bootstrap starting.
I20260806 00:58:05.830829 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 21336
I20260806 00:58:05.830971 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:05.831334 27959 external_mini_cluster.cc:1428] Setting key dae7aeabd375daddd4061c4ef0a6e6f1
I20260806 00:58:05.833935 21428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:05.834784 21428 log.cc:824] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:05.837074 21428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f: No bootstrap required, opened a new log
Aug 06 00:58:05 dist-test-slave-sb2z krb5kdc[21320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:58:05.842443 21428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548a2d8b54944254b05b20d09c21b99f" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42425 } }
I20260806 00:58:05.842751 21428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:05.842846 21428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 548a2d8b54944254b05b20d09c21b99f, State: Initialized, Role: FOLLOWER
I20260806 00:58:05.843430 21428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548a2d8b54944254b05b20d09c21b99f" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42425 } }
I20260806 00:58:05.843664 21428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:05.843779 21428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:05.843904 21428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:05.844967 21428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548a2d8b54944254b05b20d09c21b99f" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42425 } }
I20260806 00:58:05.845382 21428 leader_election.cc:302] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 548a2d8b54944254b05b20d09c21b99f; no voters:
I20260806 00:58:05.845786 21428 leader_election.cc:288] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:05.845881 21437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:05.846086 21437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [term 1 LEADER]: Becoming Leader. State: Replica: 548a2d8b54944254b05b20d09c21b99f, State: Running, Role: LEADER
I20260806 00:58:05.846505 21437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548a2d8b54944254b05b20d09c21b99f" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42425 } }
I20260806 00:58:05.847313 21428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:05.848451 21439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "548a2d8b54944254b05b20d09c21b99f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548a2d8b54944254b05b20d09c21b99f" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42425 } } }
I20260806 00:58:05.848619 21439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:05.848658 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:05.832999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56048 (local address 127.27.77.254:42425)
0806 00:58:05.833522 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:05.833538 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:05.833582 (+ 44us) server_negotiation.cc:407] Connection header received
0806 00:58:05.834359 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:05.834379 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:05.834735 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:05.835198 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:05.836081 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:05.837092 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:05.838503 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:05.839013 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:05.839168 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:05.842589 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:05.842594 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:05.842610 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:05.842641 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:05.844676 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:05.845175 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:05.845188 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:05.845194 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:05.845317 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:05.845661 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:05.845664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:05.845666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:05.846226 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:05.847096 (+ 870us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:05.847227 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 00:58:05.847600 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":160,"threads_started":1}
I20260806 00:58:05.848999 21438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 548a2d8b54944254b05b20d09c21b99f. Latest consensus state: current_term: 1 leader_uuid: "548a2d8b54944254b05b20d09c21b99f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548a2d8b54944254b05b20d09c21b99f" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 42425 } } }
I20260806 00:58:05.849109 21438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f [sys.catalog]: This master's current role is: LEADER
W20260806 00:58:05.852844 21460 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:58:05.852922 21460 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:58:05.853009 21449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:05.853785 21449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:05.860767 21449 catalog_manager.cc:1347] Generated new cluster ID: c98fa119ed714911a15bd107a40d9e62
I20260806 00:58:05.860879 21449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:05.873880 21449 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:05.875150 21449 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:05.882294 21449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f: Generated new TSK 0
I20260806 00:58:05.883181 21449 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.77.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
Entry for principal kudu/127.27.77.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab.
I20260806 00:58:05.923696 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:42425
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:06.052726 21465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:06.053026 21465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:06.053078 21465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:06.057194 21465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:06.057276 21465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:06.057339 21465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:58:06.063170 21465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.193.keytab
--principal=kudu/127.27.77.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:42425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:06.064611 21465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:06.065667 21465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:06.076246 21477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:06.076261 21475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:06.076254 21474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:06.076763 21465 server_base.cc:1034] running on GCE node
I20260806 00:58:06.077278 21465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:06.078003 21465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:06.079216 21465 hybrid_clock.cc:648] HybridClock initialized: now 1785977886079154 us; error 77 us; skew 500 ppm
Aug 06 00:58:06 dist-test-slave-sb2z krb5kdc[21320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977886, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:06.082511 21465 init.cc:377] Logged in from keytab as kudu/127.27.77.193@KRBTEST.COM (short username kudu)
I20260806 00:58:06.083668 21465 webserver.cc:460] Webserver started at http://127.27.77.193:41235/ using document root <none> and password file <none>
I20260806 00:58:06.084374 21465 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:06.084491 21465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:06.084767 21465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:06.087157 21465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "5d067807f22642e3a3bc2628aec84b2f"
format_stamp: "Formatted at 2026-08-06 00:58:06 on dist-test-slave-sb2z"
server_key: "1590febee447b16114bb6770dff30d56"
server_key_iv: "c970e41d30931fb77ef0e5dcb3b3ee95"
server_key_version: "encryptionkey@0"
I20260806 00:58:06.087834 21465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "5d067807f22642e3a3bc2628aec84b2f"
format_stamp: "Formatted at 2026-08-06 00:58:06 on dist-test-slave-sb2z"
server_key: "1590febee447b16114bb6770dff30d56"
server_key_iv: "c970e41d30931fb77ef0e5dcb3b3ee95"
server_key_version: "encryptionkey@0"
I20260806 00:58:06.092355 21465 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:58:06.095182 21488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:06.096693 21465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:58:06.096863 21465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "5d067807f22642e3a3bc2628aec84b2f"
format_stamp: "Formatted at 2026-08-06 00:58:06 on dist-test-slave-sb2z"
server_key: "1590febee447b16114bb6770dff30d56"
server_key_iv: "c970e41d30931fb77ef0e5dcb3b3ee95"
server_key_version: "encryptionkey@0"
I20260806 00:58:06.096994 21465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:06.117156 21465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:06.120980 21465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:06.121248 21465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:06.121933 21465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:58:06.122846 21465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:58:06.122924 21465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:06.123000 21465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:58:06.123049 21465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:06.133342 21465 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:33275
I20260806 00:58:06.133396 21660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:33275 every 8 connection(s)
I20260806 00:58:06.134343 21465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:58:06.142872 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 21465
I20260806 00:58:06.143062 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:58:06.143409 27959 external_mini_cluster.cc:1428] Setting key 3fbad494ce6d9b4b3e914d5af5d9277c
Aug 06 00:58:06 dist-test-slave-sb2z krb5kdc[21320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977886, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.193@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:58:06.151492 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.136604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:53801 (local address 127.27.77.254:42425)
0806 00:58:06.136761 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:06.136764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:06.137603 (+ 839us) server_negotiation.cc:407] Connection header received
0806 00:58:06.139095 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:06.139099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:06.139164 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:06.139277 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:06.140639 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.141202 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.142407 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.142782 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.142916 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.146282 (+ 3366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:06.146289 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:06.146296 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:06.146332 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:06.148383 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:06.149246 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:06.149249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:06.149250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:06.149393 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:06.149850 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:06.149857 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:06.149862 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:06.150183 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:06.150309 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:06.150952 (+ 643us) server_negotiation.cc:299] Negotiation successful
0806 00:58:06.151162 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:58:06.152315 21663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.136899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:42425 (local address 127.27.77.193:53801)
0806 00:58:06.137417 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:06.137464 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:06.138847 (+ 1383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:06.139472 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:06.139480 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:06.139929 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:06.140475 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.140489 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.141349 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.141358 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.142260 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.142270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.143070 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.143075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.143319 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.144199 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:06.144222 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:06.146083 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:06.148519 (+ 2436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:06.148527 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:06.148550 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:06.149093 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:06.149559 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:06.149562 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:06.149567 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:06.149734 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:06.150306 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:06.150312 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:06.150603 (+ 291us) client_negotiation.cc:769] Sending connection context
0806 00:58:06.150882 (+ 279us) client_negotiation.cc:240] Negotiation successful
0806 00:58:06.151229 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":119,"threads_started":1}
I20260806 00:58:06.153968 21661 heartbeater.cc:347] Connected to a master server at 127.27.77.254:42425
I20260806 00:58:06.154286 21661 heartbeater.cc:464] Registering TS with master...
I20260806 00:58:06.155014 21661 heartbeater.cc:511] Master 127.27.77.254:42425 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.77.194@KRBTEST.COM; defaulting to no policy
I20260806 00:58:06.157035 21379 ts_manager.cc:194] Registered new tserver with Master: 5d067807f22642e3a3bc2628aec84b2f (127.27.77.193:33275)
I20260806 00:58:06.158304 21379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.193@KRBTEST.COM'} at 127.27.77.193:53801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
Entry for principal kudu/127.27.77.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab.
I20260806 00:58:06.179989 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:42425
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:06.300774 21666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:06.301203 21666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:06.301301 21666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:06.305608 21666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:06.305720 21666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:06.305819 21666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:58:06.311466 21666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.194.keytab
--principal=kudu/127.27.77.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:42425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:06.312716 21666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:06.313838 21666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:06.324579 21680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:06.324592 21678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:06.324586 21677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:06.324990 21666 server_base.cc:1034] running on GCE node
I20260806 00:58:06.325618 21666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:06.326345 21666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:06.327637 21666 hybrid_clock.cc:648] HybridClock initialized: now 1785977886327623 us; error 43 us; skew 500 ppm
Aug 06 00:58:06 dist-test-slave-sb2z krb5kdc[21320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977886, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:06.330663 21666 init.cc:377] Logged in from keytab as kudu/127.27.77.194@KRBTEST.COM (short username kudu)
I20260806 00:58:06.331799 21666 webserver.cc:460] Webserver started at http://127.27.77.194:36727/ using document root <none> and password file <none>
I20260806 00:58:06.332412 21666 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:06.332511 21666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:06.332751 21666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:06.334934 21666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "04b89029202a4e34a08315b0e8848cd1"
format_stamp: "Formatted at 2026-08-06 00:58:06 on dist-test-slave-sb2z"
server_key: "fd596013e14465b853368b653e7dac6d"
server_key_iv: "cf1573e6d0b860364559c9d36f0914db"
server_key_version: "encryptionkey@0"
I20260806 00:58:06.335835 21666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "04b89029202a4e34a08315b0e8848cd1"
format_stamp: "Formatted at 2026-08-06 00:58:06 on dist-test-slave-sb2z"
server_key: "fd596013e14465b853368b653e7dac6d"
server_key_iv: "cf1573e6d0b860364559c9d36f0914db"
server_key_version: "encryptionkey@0"
I20260806 00:58:06.340308 21666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:58:06.343072 21692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:06.344578 21666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:06.344729 21666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "04b89029202a4e34a08315b0e8848cd1"
format_stamp: "Formatted at 2026-08-06 00:58:06 on dist-test-slave-sb2z"
server_key: "fd596013e14465b853368b653e7dac6d"
server_key_iv: "cf1573e6d0b860364559c9d36f0914db"
server_key_version: "encryptionkey@0"
I20260806 00:58:06.344852 21666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:06.373255 21666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:06.377034 21666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:06.377288 21666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:06.377964 21666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:58:06.378894 21666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:58:06.378957 21666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:06.378999 21666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:58:06.379014 21666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:06.389204 21666 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:42043
I20260806 00:58:06.389247 21859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:42043 every 8 connection(s)
I20260806 00:58:06.390213 21666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:58:06.399170 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 21666
I20260806 00:58:06.399334 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:58:06.399755 27959 external_mini_cluster.cc:1428] Setting key d7734a39cb6e4f92791ca14f14578647
Aug 06 00:58:06 dist-test-slave-sb2z krb5kdc[21320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977886, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.194@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:58:06.407169 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.392449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:54789 (local address 127.27.77.254:42425)
0806 00:58:06.392630 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:06.392636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:06.393710 (+ 1074us) server_negotiation.cc:407] Connection header received
0806 00:58:06.395299 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:06.395302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:06.395360 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:06.395488 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:06.397021 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.397640 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.399064 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.399478 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.399639 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.402819 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:06.402823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:06.402826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:06.402855 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:06.404720 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:06.405310 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:06.405314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:06.405318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:06.405420 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:06.405879 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:06.405882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:06.405883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:06.406107 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:06.406207 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:06.406904 (+ 697us) server_negotiation.cc:299] Negotiation successful
0806 00:58:06.407033 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 00:58:06.408533 21863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.392953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:42425 (local address 127.27.77.194:54789)
0806 00:58:06.393526 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:06.393578 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:06.395036 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:06.395777 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:06.395785 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:06.396209 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:06.396809 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.396826 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.397803 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.397806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.398886 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.398898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.399693 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.399696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.399915 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.400562 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:06.400580 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:06.402622 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:06.404855 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:06.404861 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:06.404875 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:06.405198 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:06.405535 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:06.405538 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:06.405540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:06.405743 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:06.406238 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:06.406248 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:06.406611 (+ 363us) client_negotiation.cc:769] Sending connection context
0806 00:58:06.406837 (+ 226us) client_negotiation.cc:240] Negotiation successful
0806 00:58:06.407296 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":162,"threads_started":1}
I20260806 00:58:06.409806 21861 heartbeater.cc:347] Connected to a master server at 127.27.77.254:42425
I20260806 00:58:06.410121 21861 heartbeater.cc:464] Registering TS with master...
I20260806 00:58:06.410818 21861 heartbeater.cc:511] Master 127.27.77.254:42425 requested a full tablet report, sending...
I20260806 00:58:06.412155 21379 ts_manager.cc:194] Registered new tserver with Master: 04b89029202a4e34a08315b0e8848cd1 (127.27.77.194:42043)
WARNING: no policy specified for kudu/127.27.77.195@KRBTEST.COM; defaulting to no policy
I20260806 00:58:06.412801 21379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.194@KRBTEST.COM'} at 127.27.77.194:54789
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
Entry for principal kudu/127.27.77.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab.
I20260806 00:58:06.435293 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:42425
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:06.553768 21869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:06.554049 21869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:06.554095 21869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:06.558143 21869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:06.558207 21869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:06.558267 21869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:58:06.563766 21869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.195.keytab
--principal=kudu/127.27.77.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:42425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:06.564934 21869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:06.566051 21869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:06.576275 21877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:06.576303 21875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:06.576275 21874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:06.576640 21869 server_base.cc:1034] running on GCE node
I20260806 00:58:06.577090 21869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:06.577782 21869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:06.579010 21869 hybrid_clock.cc:648] HybridClock initialized: now 1785977886578985 us; error 39 us; skew 500 ppm
Aug 06 00:58:06 dist-test-slave-sb2z krb5kdc[21320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977886, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:06.582448 21869 init.cc:377] Logged in from keytab as kudu/127.27.77.195@KRBTEST.COM (short username kudu)
I20260806 00:58:06.583576 21869 webserver.cc:460] Webserver started at http://127.27.77.195:34995/ using document root <none> and password file <none>
I20260806 00:58:06.584267 21869 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:06.584370 21869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:06.584611 21869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:06.586841 21869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "9d83b677e6ce4a438987f33e8977cb66"
format_stamp: "Formatted at 2026-08-06 00:58:06 on dist-test-slave-sb2z"
server_key: "3c810690e8e2355185783c05097f6226"
server_key_iv: "e5f134ec2e55b11b8691ba7e1f702789"
server_key_version: "encryptionkey@0"
I20260806 00:58:06.587459 21869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "9d83b677e6ce4a438987f33e8977cb66"
format_stamp: "Formatted at 2026-08-06 00:58:06 on dist-test-slave-sb2z"
server_key: "3c810690e8e2355185783c05097f6226"
server_key_iv: "e5f134ec2e55b11b8691ba7e1f702789"
server_key_version: "encryptionkey@0"
I20260806 00:58:06.591814 21869 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:58:06.594669 21886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:06.596122 21869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:06.596287 21869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "9d83b677e6ce4a438987f33e8977cb66"
format_stamp: "Formatted at 2026-08-06 00:58:06 on dist-test-slave-sb2z"
server_key: "3c810690e8e2355185783c05097f6226"
server_key_iv: "e5f134ec2e55b11b8691ba7e1f702789"
server_key_version: "encryptionkey@0"
I20260806 00:58:06.596421 21869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:06.609287 21869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:06.612941 21869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:06.613152 21869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:06.613785 21869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:58:06.614681 21869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:58:06.614756 21869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:06.614835 21869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:58:06.614889 21869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:06.625087 21869 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:38219
I20260806 00:58:06.625116 22042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:38219 every 8 connection(s)
I20260806 00:58:06.626060 21869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:58:06.633371 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 21869
I20260806 00:58:06.633518 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:58:06.633805 27959 external_mini_cluster.cc:1428] Setting key 16ab2cbac2c81f7baf52162f2355480c
Aug 06 00:58:06 dist-test-slave-sb2z krb5kdc[21320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977886, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.195@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:58:06.641301 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.628056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:43331 (local address 127.27.77.254:42425)
0806 00:58:06.628210 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:06.628213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:06.629075 (+ 862us) server_negotiation.cc:407] Connection header received
0806 00:58:06.630451 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:06.630455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:06.630514 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:06.630627 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:06.631997 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.632563 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.633828 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.634161 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.634281 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.637458 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:06.637461 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:06.637463 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:06.637490 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:06.639006 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:06.639581 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:06.639588 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:06.639593 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:06.639695 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:06.640137 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:06.640140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:06.640141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:06.640348 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:06.640443 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:06.641012 (+ 569us) server_negotiation.cc:299] Negotiation successful
0806 00:58:06.641169 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 00:58:06.642292 22047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.628440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:42425 (local address 127.27.77.195:43331)
0806 00:58:06.628922 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:06.628966 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:06.630233 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:06.630819 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:06.630826 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:06.631179 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:06.631835 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.631855 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.632704 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.632708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.633698 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.633708 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.634352 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.634358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.634568 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.635274 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:06.635296 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:06.637287 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:06.639146 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:06.639152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:06.639166 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:06.639432 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:06.639818 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:06.639821 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:06.639823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:06.640003 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:06.640479 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:06.640484 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:06.640763 (+ 279us) client_negotiation.cc:769] Sending connection context
0806 00:58:06.640955 (+ 192us) client_negotiation.cc:240] Negotiation successful
0806 00:58:06.641400 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":135,"threads_started":1}
I20260806 00:58:06.643633 22043 heartbeater.cc:347] Connected to a master server at 127.27.77.254:42425
I20260806 00:58:06.643913 22043 heartbeater.cc:464] Registering TS with master...
I20260806 00:58:06.644520 22043 heartbeater.cc:511] Master 127.27.77.254:42425 requested a full tablet report, sending...
I20260806 00:58:06.645656 21379 ts_manager.cc:194] Registered new tserver with Master: 9d83b677e6ce4a438987f33e8977cb66 (127.27.77.195:38219)
I20260806 00:58:06.646219 21379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.77.195@KRBTEST.COM'} at 127.27.77.195:43331
I20260806 00:58:06.648581 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:58:06.659863 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.651074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56058 (local address 127.27.77.254:42425)
0806 00:58:06.651243 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:06.651247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:06.651478 (+ 231us) server_negotiation.cc:407] Connection header received
0806 00:58:06.651657 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:06.651660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:06.651709 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:06.651821 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:06.652868 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.653439 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.654448 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.654680 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.654773 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.656396 (+ 1623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:06.656399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:06.656402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:06.656427 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:06.657895 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:06.658434 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:06.658438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:06.658439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:06.658531 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:06.658836 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:06.658838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:06.658839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:06.659119 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:06.659241 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:06.659515 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 00:58:06.659727 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 00:58:06.663563 21379 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:58:06.666679 21379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:58:06.683286 22062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.676305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:33275 (local address 127.0.0.1:49446)
0806 00:58:06.677110 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:06.677165 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:06.677293 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:06.678156 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:06.678162 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:06.678186 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:06.678364 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.678373 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.679966 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.679970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.681424 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.681438 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.682665 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.682669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.682796 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.682842 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 00:58:06.682968 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 00:58:06.683112 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":86,"threads_started":1}
I20260806 00:58:06.683321 22065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.676439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49446 (local address 127.27.77.193:33275)
0806 00:58:06.677627 (+ 1188us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:06.677634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:06.677660 (+ 26us) server_negotiation.cc:407] Connection header received
0806 00:58:06.677748 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:06.677756 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:06.677983 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:06.678531 (+ 548us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:06.678694 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.679819 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.681612 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.682509 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.682665 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.682833 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:06.683035 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 00:58:06.683146 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1052,"thread_start_us":111,"threads_started":1}
I20260806 00:58:06.684499 22064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.676429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:42043 (local address 127.0.0.1:53310)
0806 00:58:06.677872 (+ 1443us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:06.677885 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:06.677970 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:06.679869 (+ 1899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:06.679872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:06.679893 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:06.680006 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.680013 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.681552 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.681559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.682468 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.682478 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.683985 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.683989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.684118 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.684145 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 00:58:06.684263 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 00:58:06.684342 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1376,"mutex_wait_us":1,"thread_start_us":131,"threads_started":1}
I20260806 00:58:06.684840 22067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.676666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53310 (local address 127.27.77.194:42043)
0806 00:58:06.679244 (+ 2578us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:06.679262 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:06.679325 (+ 63us) server_negotiation.cc:407] Connection header received
0806 00:58:06.679421 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:06.679427 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:06.679684 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:06.680185 (+ 501us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:06.680387 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.681388 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.682618 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.683836 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.683987 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.684138 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:06.684298 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 00:58:06.684445 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2392,"thread_start_us":163,"threads_started":1}
I20260806 00:58:06.685007 22070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.677805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:38219 (local address 127.0.0.1:43710)
0806 00:58:06.678179 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:06.678191 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:06.678267 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:06.679135 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:06.679138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:06.679156 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:06.679249 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.679255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.681579 (+ 2324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.681584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.682740 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.682751 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.684636 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.684640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.684725 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.684749 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 00:58:06.684820 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 00:58:06.684884 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":33,"thread_start_us":98,"threads_started":1}
I20260806 00:58:06.685523 22069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.677412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43710 (local address 127.27.77.195:38219)
0806 00:58:06.678523 (+ 1111us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:06.678535 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:06.678558 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:58:06.678636 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:06.678641 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:06.678825 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:06.678971 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:06.679380 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.681412 (+ 2032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.682875 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.684311 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.684422 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.684569 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:06.685279 (+ 710us) server_negotiation.cc:299] Negotiation successful
0806 00:58:06.685377 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":998,"thread_start_us":141,"threads_started":1}
I20260806 00:58:06.686884 21557 tablet_service.cc:1467] Processing CreateTablet for tablet 394dc5ead1dd45f2b5ab34cb4e8eeb7b (DEFAULT_TABLE table=test-table [id=b3cb89fdeae8438389ffa12bde499c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:58:06.688125 21557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 394dc5ead1dd45f2b5ab34cb4e8eeb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:06.687749 21761 tablet_service.cc:1467] Processing CreateTablet for tablet 394dc5ead1dd45f2b5ab34cb4e8eeb7b (DEFAULT_TABLE table=test-table [id=b3cb89fdeae8438389ffa12bde499c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:58:06.688292 21953 tablet_service.cc:1467] Processing CreateTablet for tablet 394dc5ead1dd45f2b5ab34cb4e8eeb7b (DEFAULT_TABLE table=test-table [id=b3cb89fdeae8438389ffa12bde499c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:58:06.688983 21761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 394dc5ead1dd45f2b5ab34cb4e8eeb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:06.689395 21953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 394dc5ead1dd45f2b5ab34cb4e8eeb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:06.694659 22072 tablet_bootstrap.cc:492] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f: Bootstrap starting.
I20260806 00:58:06.696997 22074 tablet_bootstrap.cc:492] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1: Bootstrap starting.
I20260806 00:58:06.697053 22072 tablet_bootstrap.cc:654] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:06.698625 22075 tablet_bootstrap.cc:492] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66: Bootstrap starting.
I20260806 00:58:06.698796 22072 log.cc:824] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:06.699419 22074 tablet_bootstrap.cc:654] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:06.700935 22074 log.cc:824] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:06.701193 22075 tablet_bootstrap.cc:654] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:06.701567 22072 tablet_bootstrap.cc:492] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f: No bootstrap required, opened a new log
I20260806 00:58:06.701845 22072 ts_tablet_manager.cc:1397] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260806 00:58:06.702734 22075 log.cc:824] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:06.703475 22074 tablet_bootstrap.cc:492] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1: No bootstrap required, opened a new log
I20260806 00:58:06.703719 22074 ts_tablet_manager.cc:1397] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20260806 00:58:06.704768 22075 tablet_bootstrap.cc:492] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66: No bootstrap required, opened a new log
I20260806 00:58:06.704973 22075 ts_tablet_manager.cc:1397] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260806 00:58:06.706666 22072 raft_consensus.cc:348] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d067807f22642e3a3bc2628aec84b2f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 33275 } } peers { permanent_uuid: "04b89029202a4e34a08315b0e8848cd1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42043 } } peers { permanent_uuid: "9d83b677e6ce4a438987f33e8977cb66" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 38219 } }
I20260806 00:58:06.706979 22072 raft_consensus.cc:374] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:06.707041 22072 raft_consensus.cc:729] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d067807f22642e3a3bc2628aec84b2f, State: Initialized, Role: FOLLOWER
I20260806 00:58:06.707547 22072 consensus_queue.cc:260] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d067807f22642e3a3bc2628aec84b2f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 33275 } } peers { permanent_uuid: "04b89029202a4e34a08315b0e8848cd1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42043 } } peers { permanent_uuid: "9d83b677e6ce4a438987f33e8977cb66" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 38219 } }
I20260806 00:58:06.707875 22074 raft_consensus.cc:348] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d067807f22642e3a3bc2628aec84b2f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 33275 } } peers { permanent_uuid: "04b89029202a4e34a08315b0e8848cd1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42043 } } peers { permanent_uuid: "9d83b677e6ce4a438987f33e8977cb66" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 38219 } }
I20260806 00:58:06.708148 22074 raft_consensus.cc:374] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:06.708206 22074 raft_consensus.cc:729] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04b89029202a4e34a08315b0e8848cd1, State: Initialized, Role: FOLLOWER
I20260806 00:58:06.708508 21661 heartbeater.cc:503] Master 127.27.77.254:42425 was elected leader, sending a full tablet report...
I20260806 00:58:06.708629 22074 consensus_queue.cc:260] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d067807f22642e3a3bc2628aec84b2f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 33275 } } peers { permanent_uuid: "04b89029202a4e34a08315b0e8848cd1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42043 } } peers { permanent_uuid: "9d83b677e6ce4a438987f33e8977cb66" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 38219 } }
I20260806 00:58:06.708819 22075 raft_consensus.cc:348] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d067807f22642e3a3bc2628aec84b2f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 33275 } } peers { permanent_uuid: "04b89029202a4e34a08315b0e8848cd1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42043 } } peers { permanent_uuid: "9d83b677e6ce4a438987f33e8977cb66" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 38219 } }
I20260806 00:58:06.709159 22075 raft_consensus.cc:374] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:06.709239 22075 raft_consensus.cc:729] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d83b677e6ce4a438987f33e8977cb66, State: Initialized, Role: FOLLOWER
I20260806 00:58:06.709542 21861 heartbeater.cc:503] Master 127.27.77.254:42425 was elected leader, sending a full tablet report...
I20260806 00:58:06.709753 22075 consensus_queue.cc:260] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d067807f22642e3a3bc2628aec84b2f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 33275 } } peers { permanent_uuid: "04b89029202a4e34a08315b0e8848cd1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42043 } } peers { permanent_uuid: "9d83b677e6ce4a438987f33e8977cb66" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 38219 } }
I20260806 00:58:06.709875 22072 ts_tablet_manager.cc:1428] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 00:58:06.710446 22043 heartbeater.cc:503] Master 127.27.77.254:42425 was elected leader, sending a full tablet report...
I20260806 00:58:06.710611 22074 ts_tablet_manager.cc:1428] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260806 00:58:06.711362 22075 ts_tablet_manager.cc:1428] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260806 00:58:06.877210 22045 tablet.cc:2369] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:58:06.886039 21662 tablet.cc:2369] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:58:06.891594 21862 tablet.cc:2369] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:58:06.915644 22080 raft_consensus.cc:482] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:58:06.915880 22080 raft_consensus.cc:504] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d067807f22642e3a3bc2628aec84b2f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 33275 } } peers { permanent_uuid: "04b89029202a4e34a08315b0e8848cd1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42043 } } peers { permanent_uuid: "9d83b677e6ce4a438987f33e8977cb66" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 38219 } }
I20260806 00:58:06.917125 22080 leader_election.cc:288] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d067807f22642e3a3bc2628aec84b2f (127.27.77.193:33275), 9d83b677e6ce4a438987f33e8977cb66 (127.27.77.195:38219)
I20260806 00:58:06.921728 22065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.917567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:41799 (local address 127.27.77.193:33275)
0806 00:58:06.917718 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:06.917723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:06.918075 (+ 352us) server_negotiation.cc:407] Connection header received
0806 00:58:06.918240 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:06.918248 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:06.918300 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:06.918399 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:06.918721 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.919278 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.920547 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.921049 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.921208 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.921268 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:06.921480 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 00:58:06.921576 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:58:06.921742 22069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.917579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:42925 (local address 127.27.77.195:38219)
0806 00:58:06.917723 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:06.917728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:06.917900 (+ 172us) server_negotiation.cc:407] Connection header received
0806 00:58:06.918087 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:06.918090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:06.918134 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:06.918222 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:06.918523 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.919144 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.920421 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.920834 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.921009 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.921078 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:06.921524 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 00:58:06.921609 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:58:06.921759 22082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.917427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:33275 (local address 127.27.77.194:41799)
0806 00:58:06.917937 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:06.917952 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:06.918108 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:06.918427 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:06.918430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:06.918449 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:06.918595 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.918601 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.919409 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.919415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.920391 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.920403 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.921199 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.921203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.921319 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.921348 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 00:58:06.921457 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 00:58:06.921561 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":4,"thread_start_us":69,"threads_started":1}
I20260806 00:58:06.921772 22083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.917418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:38219 (local address 127.27.77.194:42925)
0806 00:58:06.917791 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:06.917804 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:06.917965 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:06.918248 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:06.918251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:06.918272 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:06.918408 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.918414 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.919288 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.919295 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.920273 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:06.920285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.921009 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.921012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:06.921120 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.921151 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 00:58:06.921498 (+ 347us) client_negotiation.cc:240] Negotiation successful
0806 00:58:06.921603 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":1,"thread_start_us":97,"threads_started":1}
I20260806 00:58:06.922569 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "394dc5ead1dd45f2b5ab34cb4e8eeb7b" candidate_uuid: "04b89029202a4e34a08315b0e8848cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d067807f22642e3a3bc2628aec84b2f" is_pre_election: true
I20260806 00:58:06.922858 21984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "394dc5ead1dd45f2b5ab34cb4e8eeb7b" candidate_uuid: "04b89029202a4e34a08315b0e8848cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d83b677e6ce4a438987f33e8977cb66" is_pre_election: true
I20260806 00:58:06.923019 21588 raft_consensus.cc:2454] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04b89029202a4e34a08315b0e8848cd1 in term 0.
I20260806 00:58:06.923221 21984 raft_consensus.cc:2454] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04b89029202a4e34a08315b0e8848cd1 in term 0.
I20260806 00:58:06.923823 21696 leader_election.cc:302] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 04b89029202a4e34a08315b0e8848cd1, 5d067807f22642e3a3bc2628aec84b2f, 9d83b677e6ce4a438987f33e8977cb66; no voters:
I20260806 00:58:06.924300 22080 raft_consensus.cc:2781] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:58:06.924420 22080 raft_consensus.cc:482] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:58:06.924470 22080 raft_consensus.cc:3037] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:06.925662 22080 raft_consensus.cc:504] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d067807f22642e3a3bc2628aec84b2f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 33275 } } peers { permanent_uuid: "04b89029202a4e34a08315b0e8848cd1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42043 } } peers { permanent_uuid: "9d83b677e6ce4a438987f33e8977cb66" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 38219 } }
I20260806 00:58:06.926075 22080 leader_election.cc:288] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 5d067807f22642e3a3bc2628aec84b2f (127.27.77.193:33275), 9d83b677e6ce4a438987f33e8977cb66 (127.27.77.195:38219)
I20260806 00:58:06.926645 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "394dc5ead1dd45f2b5ab34cb4e8eeb7b" candidate_uuid: "04b89029202a4e34a08315b0e8848cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d067807f22642e3a3bc2628aec84b2f"
I20260806 00:58:06.926689 21984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "394dc5ead1dd45f2b5ab34cb4e8eeb7b" candidate_uuid: "04b89029202a4e34a08315b0e8848cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d83b677e6ce4a438987f33e8977cb66"
I20260806 00:58:06.926817 21984 raft_consensus.cc:3037] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:06.926817 21588 raft_consensus.cc:3037] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:06.928579 21588 raft_consensus.cc:2454] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04b89029202a4e34a08315b0e8848cd1 in term 1.
I20260806 00:58:06.928579 21984 raft_consensus.cc:2454] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04b89029202a4e34a08315b0e8848cd1 in term 1.
I20260806 00:58:06.929041 21695 leader_election.cc:302] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04b89029202a4e34a08315b0e8848cd1, 9d83b677e6ce4a438987f33e8977cb66; no voters:
I20260806 00:58:06.929337 22080 raft_consensus.cc:2781] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:06.929612 22080 raft_consensus.cc:686] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 04b89029202a4e34a08315b0e8848cd1, State: Running, Role: LEADER
I20260806 00:58:06.930013 22080 consensus_queue.cc:237] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d067807f22642e3a3bc2628aec84b2f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 33275 } } peers { permanent_uuid: "04b89029202a4e34a08315b0e8848cd1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42043 } } peers { permanent_uuid: "9d83b677e6ce4a438987f33e8977cb66" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 38219 } }
I20260806 00:58:06.933636 21369 catalog_manager.cc:5495] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04b89029202a4e34a08315b0e8848cd1 (127.27.77.194). New cstate: current_term: 1 leader_uuid: "04b89029202a4e34a08315b0e8848cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d067807f22642e3a3bc2628aec84b2f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04b89029202a4e34a08315b0e8848cd1" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 42043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d83b677e6ce4a438987f33e8977cb66" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 38219 } health_report { overall_health: UNKNOWN } } }
I20260806 00:58:06.977885 22067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:06.973952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53316 (local address 127.27.77.194:42043)
0806 00:58:06.974108 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:06.974114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:06.974130 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:58:06.974308 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:06.974310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:06.974375 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:06.974518 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:58:06.974788 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.975302 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.976315 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:06.976553 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:06.976673 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:06.977020 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:58:06.977464 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:58:06.977609 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:06.977672 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 00:58:06.977732 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:58:06.984452 21984 raft_consensus.cc:1261] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66 [term 1 FOLLOWER]: Refusing update from remote peer 04b89029202a4e34a08315b0e8848cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:58:06.984452 21588 raft_consensus.cc:1261] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f [term 1 FOLLOWER]: Refusing update from remote peer 04b89029202a4e34a08315b0e8848cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:58:06.985292 22084 consensus_queue.cc:1035] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d83b677e6ce4a438987f33e8977cb66" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 38219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:58:06.985504 22080 consensus_queue.cc:1035] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d067807f22642e3a3bc2628aec84b2f" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:58:06.995332 22088 mvcc.cc:205] Tried to move back new op lower bound from 7315365425079840768 to 7315365424868540416. Current Snapshot: MvccSnapshot[applied={T|T < 7315365425079840768}]
I20260806 00:58:07.004580 21369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:58:07.004801 21369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:58:07.007622 21369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 548a2d8b54944254b05b20d09c21b99f: Sending DeleteTablet for 3 replicas of tablet 394dc5ead1dd45f2b5ab34cb4e8eeb7b
I20260806 00:58:07.008641 21953 tablet_service.cc:1514] Processing DeleteTablet for tablet 394dc5ead1dd45f2b5ab34cb4e8eeb7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:58:07 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:43710
I20260806 00:58:07.008636 21761 tablet_service.cc:1514] Processing DeleteTablet for tablet 394dc5ead1dd45f2b5ab34cb4e8eeb7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:58:07 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:53310
I20260806 00:58:07.009096 21557 tablet_service.cc:1514] Processing DeleteTablet for tablet 394dc5ead1dd45f2b5ab34cb4e8eeb7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:58:07 UTC) from {username='kudu', principal='kudu/127.27.77.254@KRBTEST.COM'} at 127.0.0.1:49446
I20260806 00:58:07.009295 22102 tablet_replica.cc:330] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1: stopping tablet replica
I20260806 00:58:07.009670 22104 tablet_replica.cc:330] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f: stopping tablet replica
I20260806 00:58:07.009714 22102 raft_consensus.cc:2229] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:58:07.009994 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 21465
I20260806 00:58:07.010152 22104 raft_consensus.cc:2229] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 5d067807f22642e3a3bc2628aec84b2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:58:07.010185 22102 raft_consensus.cc:2258] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:58:07.010703 22101 tablet_replica.cc:330] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66: stopping tablet replica
I20260806 00:58:07.010995 22101 raft_consensus.cc:2229] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:58:07.011303 22101 raft_consensus.cc:2258] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:58:07.016541 22102 ts_tablet_manager.cc:1905] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:58:07.017776 22101 ts_tablet_manager.cc:1905] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:58:07.019817 22102 ts_tablet_manager.cc:1918] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:58:07.020018 22102 log.cc:1196] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/394dc5ead1dd45f2b5ab34cb4e8eeb7b
I20260806 00:58:07.020423 22102 ts_tablet_manager.cc:1939] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 04b89029202a4e34a08315b0e8848cd1: Deleting consensus metadata
I20260806 00:58:07.020833 22101 ts_tablet_manager.cc:1918] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:58:07.020970 22101 log.cc:1196] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/394dc5ead1dd45f2b5ab34cb4e8eeb7b
I20260806 00:58:07.021332 22101 ts_tablet_manager.cc:1939] T 394dc5ead1dd45f2b5ab34cb4e8eeb7b P 9d83b677e6ce4a438987f33e8977cb66: Deleting consensus metadata
I20260806 00:58:07.022250 21357 catalog_manager.cc:4841] TS 04b89029202a4e34a08315b0e8848cd1 (127.27.77.194:42043): tablet 394dc5ead1dd45f2b5ab34cb4e8eeb7b (table test-table [id=b3cb89fdeae8438389ffa12bde499c6e]) successfully deleted
I20260806 00:58:07.022503 21357 catalog_manager.cc:4841] TS 9d83b677e6ce4a438987f33e8977cb66 (127.27.77.195:38219): tablet 394dc5ead1dd45f2b5ab34cb4e8eeb7b (table test-table [id=b3cb89fdeae8438389ffa12bde499c6e]) successfully deleted
W20260806 00:58:07.026803 21358 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:33275 (error 108)
I20260806 00:58:07.027837 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 21666
I20260806 00:58:07.027993 22070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:07.027658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:33275 (local address 127.0.0.1:49460)
0806 00:58:07.027815 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:07.027893 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 00:58:07.042809 21358 catalog_manager.cc:4584] TS 5d067807f22642e3a3bc2628aec84b2f (127.27.77.193:33275): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 394dc5ead1dd45f2b5ab34cb4e8eeb7b: Network error: Client connection negotiation failed: client connection to 127.27.77.193:33275: connect: Connection refused (error 111)
I20260806 00:58:07.049603 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 21869
I20260806 00:58:07.059952 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 21336
2026-08-06T00:58:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2800 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T00:58:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:07Z Disabled control of system clock
I20260806 00:58:07.100355 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:35645
--webserver_interface=127.27.77.254
--webserver_port=0
--builtin_ntp_servers=127.27.77.212:33785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:35645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:58:07.214265 22110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:07.214543 22110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:07.214579 22110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:07.218757 22110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:07.218822 22110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:07.218837 22110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:07.218853 22110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:07.218868 22110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:07.223667 22110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:33785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:35645
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:35645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:07.224723 22110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:07.226194 22110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:07.235184 22117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:07.235208 22116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:07.235184 22119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:07.235278 22110 server_base.cc:1034] running on GCE node
I20260806 00:58:07.236128 22110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:07.237313 22110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:07.238489 22110 hybrid_clock.cc:648] HybridClock initialized: now 1785977887238463 us; error 40 us; skew 500 ppm
I20260806 00:58:07.244953 22110 webserver.cc:460] Webserver started at http://127.27.77.254:33281/ using document root <none> and password file <none>
I20260806 00:58:07.245596 22110 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:07.245676 22110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:07.245910 22110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:07.248185 22110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "cadf89db55ff4329b03911729d86ea05"
format_stamp: "Formatted at 2026-08-06 00:58:07 on dist-test-slave-sb2z"
server_key: "71936254304aee6b269ee357582e1c6a"
server_key_iv: "f330287d46cf3ab2a44996da27f89dbc"
server_key_version: "encryptionkey@0"
I20260806 00:58:07.248799 22110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "cadf89db55ff4329b03911729d86ea05"
format_stamp: "Formatted at 2026-08-06 00:58:07 on dist-test-slave-sb2z"
server_key: "71936254304aee6b269ee357582e1c6a"
server_key_iv: "f330287d46cf3ab2a44996da27f89dbc"
server_key_version: "encryptionkey@0"
I20260806 00:58:07.252645 22110 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:58:07.255218 22132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:07.256577 22110 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:58:07.256734 22110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "cadf89db55ff4329b03911729d86ea05"
format_stamp: "Formatted at 2026-08-06 00:58:07 on dist-test-slave-sb2z"
server_key: "71936254304aee6b269ee357582e1c6a"
server_key_iv: "f330287d46cf3ab2a44996da27f89dbc"
server_key_version: "encryptionkey@0"
I20260806 00:58:07.256860 22110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:07.272127 22110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:07.272953 22110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:07.273180 22110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:07.280860 22110 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:35645
I20260806 00:58:07.280889 22199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:35645 every 8 connection(s)
I20260806 00:58:07.281906 22110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:07.285243 22200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:07.287017 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 22110
I20260806 00:58:07.287220 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:07.287515 27959 external_mini_cluster.cc:1428] Setting key 5bb9487e1a60c4410cb4c97d72043640
I20260806 00:58:07.291695 22200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05: Bootstrap starting.
I20260806 00:58:07.293941 22200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:07.294709 22200 log.cc:824] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:07.296998 22200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05: No bootstrap required, opened a new log
I20260806 00:58:07.297494 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:07.289099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52662 (local address 127.27.77.254:35645)
0806 00:58:07.289842 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:07.289858 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:07.289895 (+ 37us) server_negotiation.cc:407] Connection header received
0806 00:58:07.290773 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:07.290796 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:07.291198 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:07.291599 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:07.292050 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:07.293009 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:07.293899 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:07.294450 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:07.294650 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:07.295587 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:07.295595 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:07.295615 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:07.296006 (+ 391us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:07.296068 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:07.296089 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:07.296173 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:07.296387 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 00:58:07.296709 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":144,"threads_started":1}
I20260806 00:58:07.301079 22200 raft_consensus.cc:348] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cadf89db55ff4329b03911729d86ea05" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 35645 } }
I20260806 00:58:07.301340 22200 raft_consensus.cc:374] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:07.301422 22200 raft_consensus.cc:729] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cadf89db55ff4329b03911729d86ea05, State: Initialized, Role: FOLLOWER
I20260806 00:58:07.301851 22200 consensus_queue.cc:260] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cadf89db55ff4329b03911729d86ea05" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 35645 } }
I20260806 00:58:07.302006 22200 raft_consensus.cc:388] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:07.302099 22200 raft_consensus.cc:482] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:07.302222 22200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:07.303400 22200 raft_consensus.cc:504] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cadf89db55ff4329b03911729d86ea05" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 35645 } }
I20260806 00:58:07.303776 22200 leader_election.cc:302] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cadf89db55ff4329b03911729d86ea05; no voters:
I20260806 00:58:07.304144 22200 leader_election.cc:288] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:07.304282 22209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:07.304549 22209 raft_consensus.cc:686] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [term 1 LEADER]: Becoming Leader. State: Replica: cadf89db55ff4329b03911729d86ea05, State: Running, Role: LEADER
I20260806 00:58:07.304867 22209 consensus_queue.cc:237] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cadf89db55ff4329b03911729d86ea05" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 35645 } }
I20260806 00:58:07.305416 22200 sys_catalog.cc:564] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:07.306720 22211 sys_catalog.cc:455] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cadf89db55ff4329b03911729d86ea05. Latest consensus state: current_term: 1 leader_uuid: "cadf89db55ff4329b03911729d86ea05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cadf89db55ff4329b03911729d86ea05" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 35645 } } }
I20260806 00:58:07.306732 22210 sys_catalog.cc:455] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cadf89db55ff4329b03911729d86ea05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cadf89db55ff4329b03911729d86ea05" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 35645 } } }
I20260806 00:58:07.306833 22211 sys_catalog.cc:458] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:07.306854 22210 sys_catalog.cc:458] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:07.307322 22224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:07.310635 22224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:07.316903 22224 catalog_manager.cc:1347] Generated new cluster ID: 52988db10e8b4e75a17e1dc70912184a
I20260806 00:58:07.317041 22224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:07.330989 22224 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:07.332110 22224 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:07.345295 22224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05: Generated new TSK 0
I20260806 00:58:07.346289 22224 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:07.360641 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.193:0
--local_ip_for_outbound_sockets=127.27.77.193
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:35645
--builtin_ntp_servers=127.27.77.212:33785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:58:07.480744 22243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:07.481102 22243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:07.481186 22243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:07.485353 22243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:07.485457 22243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:07.485555 22243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.193
I20260806 00:58:07.490548 22243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:33785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.77.193
--webserver_port=0
--tserver_master_addrs=127.27.77.254:35645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.193
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:07.491760 22243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:07.493175 22243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:07.503088 22252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:07.503165 22254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:07.503155 22251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:07.503316 22243 server_base.cc:1034] running on GCE node
I20260806 00:58:07.503880 22243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:07.504618 22243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:07.505842 22243 hybrid_clock.cc:648] HybridClock initialized: now 1785977887505803 us; error 48 us; skew 500 ppm
I20260806 00:58:07.508353 22243 webserver.cc:460] Webserver started at http://127.27.77.193:35359/ using document root <none> and password file <none>
I20260806 00:58:07.508981 22243 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:07.509076 22243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:07.509294 22243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:07.511381 22243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/data/instance:
uuid: "9c938ee6c8584eab855c62e8b0940ed2"
format_stamp: "Formatted at 2026-08-06 00:58:07 on dist-test-slave-sb2z"
server_key: "27dcbcab276d813c4894d589d4369112"
server_key_iv: "fda116b9bd98b9f8b285a496d6f7ab93"
server_key_version: "encryptionkey@0"
I20260806 00:58:07.512055 22243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance:
uuid: "9c938ee6c8584eab855c62e8b0940ed2"
format_stamp: "Formatted at 2026-08-06 00:58:07 on dist-test-slave-sb2z"
server_key: "27dcbcab276d813c4894d589d4369112"
server_key_iv: "fda116b9bd98b9f8b285a496d6f7ab93"
server_key_version: "encryptionkey@0"
I20260806 00:58:07.515920 22243 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:58:07.518445 22262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:07.519826 22243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:07.519973 22243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/wal
uuid: "9c938ee6c8584eab855c62e8b0940ed2"
format_stamp: "Formatted at 2026-08-06 00:58:07 on dist-test-slave-sb2z"
server_key: "27dcbcab276d813c4894d589d4369112"
server_key_iv: "fda116b9bd98b9f8b285a496d6f7ab93"
server_key_version: "encryptionkey@0"
I20260806 00:58:07.520092 22243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:07.537364 22243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:07.538203 22243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:07.538424 22243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:07.539139 22243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:58:07.540181 22243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:58:07.540257 22243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:07.540336 22243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:58:07.540386 22243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:07.550083 22243 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.193:38175
I20260806 00:58:07.550164 22429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.193:38175 every 8 connection(s)
I20260806 00:58:07.551066 22243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/data/info.pb
I20260806 00:58:07.559317 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 22243
I20260806 00:58:07.559458 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-0/wal/instance
I20260806 00:58:07.559847 27959 external_mini_cluster.cc:1428] Setting key 0df696810d47ab1662beffa3fe1cbb38
I20260806 00:58:07.561528 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:07.553065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.193:53045 (local address 127.27.77.254:35645)
0806 00:58:07.553225 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:07.553229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:07.554444 (+ 1215us) server_negotiation.cc:407] Connection header received
0806 00:58:07.555854 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:07.555857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:07.555905 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:07.555991 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:07.557086 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:07.557623 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:07.558939 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:07.559168 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:07.559254 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:07.560763 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:07.560767 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:07.560770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:07.560786 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:07.560810 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:07.560821 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:07.560908 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:07.561231 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 00:58:07.561302 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 00:58:07.562592 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:07.553491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:35645 (local address 127.27.77.193:53045)
0806 00:58:07.554266 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:07.554319 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:07.555650 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:07.556246 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:07.556253 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:07.556342 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:07.556884 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:07.556898 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:07.557747 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:07.557749 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:07.558803 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:07.558814 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:07.559358 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:07.559363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:07.559670 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:07.560421 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:58:07.560423 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:07.560486 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:58:07.560571 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:07.560980 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:07.560992 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:07.561000 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 00:58:07.561156 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 00:58:07.561481 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":174,"threads_started":1}
I20260806 00:58:07.562794 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.194:0
--local_ip_for_outbound_sockets=127.27.77.194
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:35645
--builtin_ntp_servers=127.27.77.212:33785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:58:07.563999 22430 heartbeater.cc:347] Connected to a master server at 127.27.77.254:35645
I20260806 00:58:07.564253 22430 heartbeater.cc:464] Registering TS with master...
I20260806 00:58:07.564977 22430 heartbeater.cc:511] Master 127.27.77.254:35645 requested a full tablet report, sending...
I20260806 00:58:07.566712 22153 ts_manager.cc:194] Registered new tserver with Master: 9c938ee6c8584eab855c62e8b0940ed2 (127.27.77.193:38175)
I20260806 00:58:07.568048 22153 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.77.193:53045
W20260806 00:58:07.686759 22435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:07.687047 22435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:07.687088 22435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:07.691387 22435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:07.691457 22435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:07.691560 22435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.194
I20260806 00:58:07.696625 22435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:33785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.77.194
--webserver_port=0
--tserver_master_addrs=127.27.77.254:35645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.194
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:07.697809 22435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:07.699477 22435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:07.709877 22447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:07.709864 22451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:07.709864 22446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:07.710155 22435 server_base.cc:1034] running on GCE node
I20260806 00:58:07.710840 22435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:07.711517 22435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:07.712765 22435 hybrid_clock.cc:648] HybridClock initialized: now 1785977887712735 us; error 39 us; skew 500 ppm
I20260806 00:58:07.714792 22435 webserver.cc:460] Webserver started at http://127.27.77.194:41019/ using document root <none> and password file <none>
I20260806 00:58:07.715397 22435 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:07.715463 22435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:07.715772 22435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:07.717927 22435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/data/instance:
uuid: "210a2731d3a747cf85914d536c79b372"
format_stamp: "Formatted at 2026-08-06 00:58:07 on dist-test-slave-sb2z"
server_key: "5b008e7095e091c84f0ca45fb59a2cf3"
server_key_iv: "f0b21a349bef09b02581449723fc8929"
server_key_version: "encryptionkey@0"
I20260806 00:58:07.718523 22435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance:
uuid: "210a2731d3a747cf85914d536c79b372"
format_stamp: "Formatted at 2026-08-06 00:58:07 on dist-test-slave-sb2z"
server_key: "5b008e7095e091c84f0ca45fb59a2cf3"
server_key_iv: "f0b21a349bef09b02581449723fc8929"
server_key_version: "encryptionkey@0"
I20260806 00:58:07.722383 22435 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 00:58:07.725147 22457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:07.726488 22435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 00:58:07.726630 22435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/wal
uuid: "210a2731d3a747cf85914d536c79b372"
format_stamp: "Formatted at 2026-08-06 00:58:07 on dist-test-slave-sb2z"
server_key: "5b008e7095e091c84f0ca45fb59a2cf3"
server_key_iv: "f0b21a349bef09b02581449723fc8929"
server_key_version: "encryptionkey@0"
I20260806 00:58:07.726758 22435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:07.748692 22435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:07.749557 22435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:07.749802 22435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:07.750530 22435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:58:07.751721 22435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:58:07.751778 22435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:07.751827 22435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:58:07.751883 22435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:07.762987 22435 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.194:44741
I20260806 00:58:07.763003 22619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.194:44741 every 8 connection(s)
I20260806 00:58:07.764161 22435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/data/info.pb
I20260806 00:58:07.771654 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 22435
I20260806 00:58:07.771791 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/wal/instance
I20260806 00:58:07.772174 27959 external_mini_cluster.cc:1428] Setting key 712aa45abfcabbe265268e759fb006d9
I20260806 00:58:07.774590 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.77.195:0
--local_ip_for_outbound_sockets=127.27.77.195
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:35645
--builtin_ntp_servers=127.27.77.212:33785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:58:07.774721 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:07.766234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.194:42175 (local address 127.27.77.254:35645)
0806 00:58:07.766436 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:07.766441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:07.767595 (+ 1154us) server_negotiation.cc:407] Connection header received
0806 00:58:07.769083 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:07.769087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:07.769141 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:07.769233 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:07.770174 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:07.770742 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:07.772360 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:07.772659 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:07.772790 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:07.774128 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:07.774131 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:07.774133 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:07.774146 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:07.774173 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:07.774182 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:07.774243 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:07.774505 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 00:58:07.774564 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 00:58:07.775929 22622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:07.766712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:35645 (local address 127.27.77.194:42175)
0806 00:58:07.767417 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:07.767459 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:07.768848 (+ 1389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:07.769410 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:07.769417 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:07.769502 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:07.770016 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:07.770029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:07.770879 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:07.770882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:07.772224 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:07.772238 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:07.772863 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:07.772867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:07.773079 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:07.773820 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:58:07.773822 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:07.773879 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:58:07.773967 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:07.774275 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:07.774286 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:07.774295 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 00:58:07.774445 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 00:58:07.774806 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":149,"threads_started":1}
I20260806 00:58:07.777252 22620 heartbeater.cc:347] Connected to a master server at 127.27.77.254:35645
I20260806 00:58:07.777504 22620 heartbeater.cc:464] Registering TS with master...
I20260806 00:58:07.778051 22620 heartbeater.cc:511] Master 127.27.77.254:35645 requested a full tablet report, sending...
I20260806 00:58:07.779201 22153 ts_manager.cc:194] Registered new tserver with Master: 210a2731d3a747cf85914d536c79b372 (127.27.77.194:44741)
I20260806 00:58:07.780133 22153 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.77.194:42175
W20260806 00:58:07.893882 22623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:07.894147 22623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:07.894189 22623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:07.898150 22623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:07.898205 22623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:07.898262 22623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.77.195
I20260806 00:58:07.903156 22623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:33785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.77.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.77.195
--webserver_port=0
--tserver_master_addrs=127.27.77.254:35645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.77.195
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:07.904268 22623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:07.905738 22623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:07.915920 22628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:07.915966 22631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:07.915948 22629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:07.916280 22623 server_base.cc:1034] running on GCE node
I20260806 00:58:07.916783 22623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:07.917452 22623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:07.918630 22623 hybrid_clock.cc:648] HybridClock initialized: now 1785977887918611 us; error 36 us; skew 500 ppm
I20260806 00:58:07.920724 22623 webserver.cc:460] Webserver started at http://127.27.77.195:33227/ using document root <none> and password file <none>
I20260806 00:58:07.921370 22623 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:07.921440 22623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:07.921674 22623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:07.923933 22623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/data/instance:
uuid: "169dfec43e184121a54e8f386242c603"
format_stamp: "Formatted at 2026-08-06 00:58:07 on dist-test-slave-sb2z"
server_key: "724c4e659340513db0478adb8aec2ebd"
server_key_iv: "a412a8776757d62f0c6b4eb77436d5fe"
server_key_version: "encryptionkey@0"
I20260806 00:58:07.924521 22623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance:
uuid: "169dfec43e184121a54e8f386242c603"
format_stamp: "Formatted at 2026-08-06 00:58:07 on dist-test-slave-sb2z"
server_key: "724c4e659340513db0478adb8aec2ebd"
server_key_iv: "a412a8776757d62f0c6b4eb77436d5fe"
server_key_version: "encryptionkey@0"
I20260806 00:58:07.928469 22623 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:58:07.931223 22642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:07.932562 22623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:58:07.932689 22623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/wal
uuid: "169dfec43e184121a54e8f386242c603"
format_stamp: "Formatted at 2026-08-06 00:58:07 on dist-test-slave-sb2z"
server_key: "724c4e659340513db0478adb8aec2ebd"
server_key_iv: "a412a8776757d62f0c6b4eb77436d5fe"
server_key_version: "encryptionkey@0"
I20260806 00:58:07.932801 22623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:07.964736 22623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:07.965531 22623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:07.965751 22623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:07.966450 22623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:58:07.967468 22623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:58:07.967543 22623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:07.967588 22623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:58:07.967638 22623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:07.978174 22623 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.195:44535
I20260806 00:58:07.978215 22793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.195:44535 every 8 connection(s)
I20260806 00:58:07.979240 22623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/data/info.pb
I20260806 00:58:07.983584 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 22623
I20260806 00:58:07.983716 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/wal/instance
I20260806 00:58:07.984022 27959 external_mini_cluster.cc:1428] Setting key 5866644fb96a7b179a6da0f1a0c60497
I20260806 00:58:07.989886 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:07.981330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:56959 (local address 127.27.77.254:35645)
0806 00:58:07.981483 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:07.981487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:07.982664 (+ 1177us) server_negotiation.cc:407] Connection header received
0806 00:58:07.984509 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:07.984514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:07.984572 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:07.984660 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:07.985980 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:07.986707 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:07.987984 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:07.988205 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:07.988285 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:07.989374 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:07.989377 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:07.989380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:07.989392 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:07.989412 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:07.989420 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:07.989497 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:07.989704 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 00:58:07.989759 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:58:07.990823 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:07.981697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:35645 (local address 127.27.77.195:56959)
0806 00:58:07.982479 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:07.982524 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:07.984207 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:07.984918 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:07.984927 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:07.985041 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:07.985808 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:07.985829 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:07.986831 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:07.986834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:07.987853 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:07.987864 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:07.988340 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:07.988342 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:07.988481 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:07.989128 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:58:07.989129 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:07.989169 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:58:07.989233 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:07.989524 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:07.989532 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:07.989537 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 00:58:07.989679 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 00:58:07.989905 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":129,"threads_started":1}
I20260806 00:58:07.992198 22794 heartbeater.cc:347] Connected to a master server at 127.27.77.254:35645
I20260806 00:58:07.992471 22794 heartbeater.cc:464] Registering TS with master...
I20260806 00:58:07.993033 22794 heartbeater.cc:511] Master 127.27.77.254:35645 requested a full tablet report, sending...
I20260806 00:58:07.994220 22153 ts_manager.cc:194] Registered new tserver with Master: 169dfec43e184121a54e8f386242c603 (127.27.77.195:44535)
I20260806 00:58:07.994899 22153 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.77.195:56959
I20260806 00:58:07.998373 27959 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:58:08.004537 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:08.000606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52666 (local address 127.27.77.254:35645)
0806 00:58:08.000740 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:08.000744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:08.000872 (+ 128us) server_negotiation.cc:407] Connection header received
0806 00:58:08.000983 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:08.000985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:08.001022 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:08.001092 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:08.001438 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.002011 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.002880 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.003138 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.003224 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:08.004142 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:08.004146 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:08.004148 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:08.004159 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:08.004178 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:08.004186 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:08.004250 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:08.004353 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 00:58:08.004407 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:58:08.007818 22153 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:58:08.010908 22153 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:58:08.026587 22811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:08.019948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:38175 (local address 127.0.0.1:53266)
0806 00:58:08.020797 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:08.020810 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:08.020901 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:08.021663 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:08.021670 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:08.021714 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:08.021892 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:08.021904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.023217 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.023223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:08.025146 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:08.025153 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.026037 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.026041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:08.026148 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:08.026185 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 00:58:08.026299 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 00:58:08.026423 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":175,"spinlock_wait_cycles":5632,"thread_start_us":106,"threads_started":1}
I20260806 00:58:08.026734 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:08.021152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54626 (local address 127.27.77.194:44741)
0806 00:58:08.021769 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:08.021781 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:08.021803 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:58:08.021877 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:08.021881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:08.022063 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:08.022215 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:08.022448 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.023664 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.025405 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.026106 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.026233 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:08.026420 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:08.026472 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 00:58:08.026569 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":157,"threads_started":1}
I20260806 00:58:08.027047 22819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:08.020670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:44741 (local address 127.0.0.1:54626)
0806 00:58:08.021251 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:08.021265 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:08.021361 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:08.022215 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:08.022218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:08.022239 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:08.022319 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:08.022326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.023798 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.023801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:08.025277 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:08.025288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.026246 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.026250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:08.026335 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:08.026358 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:58:08.026420 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 00:58:08.026504 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":283,"thread_start_us":98,"threads_started":1}
I20260806 00:58:08.027226 22816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:08.020138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53266 (local address 127.27.77.193:38175)
0806 00:58:08.020671 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:08.020681 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:08.020910 (+ 229us) server_negotiation.cc:407] Connection header received
0806 00:58:08.021028 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:08.021034 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:08.021255 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:08.021450 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:08.022046 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.023100 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.025286 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.025895 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.026029 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:08.026233 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:08.026928 (+ 695us) server_negotiation.cc:299] Negotiation successful
0806 00:58:08.027050 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":90,"threads_started":1}
I20260806 00:58:08.027899 22814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:08.019728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.195:44535 (local address 127.0.0.1:48654)
0806 00:58:08.020631 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:08.020686 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:08.020823 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:08.021664 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:08.021670 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:08.021710 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:08.021879 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:08.021887 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.023381 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.023385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:08.025140 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:08.025153 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.027476 (+ 2323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.027480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:08.027607 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:08.027631 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 00:58:08.027705 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 00:58:08.027770 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":263,"spinlock_wait_cycles":4736,"thread_start_us":97,"threads_started":1}
I20260806 00:58:08.028334 22812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:08.019883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48654 (local address 127.27.77.195:44535)
0806 00:58:08.020747 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:08.020764 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:08.020798 (+ 34us) server_negotiation.cc:407] Connection header received
0806 00:58:08.021112 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:08.021120 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:08.021333 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:08.021478 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:08.022014 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.023261 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.026298 (+ 3037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.027086 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.027192 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:08.027380 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:08.028075 (+ 695us) server_negotiation.cc:299] Negotiation successful
0806 00:58:08.028176 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":109,"threads_started":1}
I20260806 00:58:08.030365 22524 tablet_service.cc:1467] Processing CreateTablet for tablet 2c099210d10f475a911225b28e747b35 (DEFAULT_TABLE table=test-table [id=4dc4b94042764077b8a33a1d8e56ab84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:58:08.030921 22706 tablet_service.cc:1467] Processing CreateTablet for tablet 2c099210d10f475a911225b28e747b35 (DEFAULT_TABLE table=test-table [id=4dc4b94042764077b8a33a1d8e56ab84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:58:08.031585 22524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c099210d10f475a911225b28e747b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:08.031948 22706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c099210d10f475a911225b28e747b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:08.037683 22333 tablet_service.cc:1467] Processing CreateTablet for tablet 2c099210d10f475a911225b28e747b35 (DEFAULT_TABLE table=test-table [id=4dc4b94042764077b8a33a1d8e56ab84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:58:08.038762 22333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c099210d10f475a911225b28e747b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:08.039430 22821 tablet_bootstrap.cc:492] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603: Bootstrap starting.
I20260806 00:58:08.040990 22823 tablet_bootstrap.cc:492] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372: Bootstrap starting.
I20260806 00:58:08.042207 22821 tablet_bootstrap.cc:654] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:08.043125 22821 log.cc:824] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:08.043663 22823 tablet_bootstrap.cc:654] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:08.044534 22823 log.cc:824] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:08.045288 22821 tablet_bootstrap.cc:492] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603: No bootstrap required, opened a new log
I20260806 00:58:08.045524 22821 ts_tablet_manager.cc:1397] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:58:08.046686 22823 tablet_bootstrap.cc:492] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372: No bootstrap required, opened a new log
I20260806 00:58:08.046880 22823 ts_tablet_manager.cc:1397] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260806 00:58:08.047870 22826 tablet_bootstrap.cc:492] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2: Bootstrap starting.
I20260806 00:58:08.049875 22826 tablet_bootstrap.cc:654] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:08.050172 22821 raft_consensus.cc:348] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169dfec43e184121a54e8f386242c603" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44535 } } peers { permanent_uuid: "9c938ee6c8584eab855c62e8b0940ed2" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38175 } } peers { permanent_uuid: "210a2731d3a747cf85914d536c79b372" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44741 } }
I20260806 00:58:08.050455 22821 raft_consensus.cc:374] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:08.050518 22821 raft_consensus.cc:729] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 169dfec43e184121a54e8f386242c603, State: Initialized, Role: FOLLOWER
I20260806 00:58:08.050829 22826 log.cc:824] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:08.050994 22821 consensus_queue.cc:260] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169dfec43e184121a54e8f386242c603" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44535 } } peers { permanent_uuid: "9c938ee6c8584eab855c62e8b0940ed2" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38175 } } peers { permanent_uuid: "210a2731d3a747cf85914d536c79b372" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44741 } }
I20260806 00:58:08.051271 22823 raft_consensus.cc:348] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169dfec43e184121a54e8f386242c603" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44535 } } peers { permanent_uuid: "9c938ee6c8584eab855c62e8b0940ed2" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38175 } } peers { permanent_uuid: "210a2731d3a747cf85914d536c79b372" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44741 } }
I20260806 00:58:08.051548 22823 raft_consensus.cc:374] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:08.051613 22823 raft_consensus.cc:729] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210a2731d3a747cf85914d536c79b372, State: Initialized, Role: FOLLOWER
I20260806 00:58:08.051891 22794 heartbeater.cc:503] Master 127.27.77.254:35645 was elected leader, sending a full tablet report...
I20260806 00:58:08.052052 22823 consensus_queue.cc:260] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169dfec43e184121a54e8f386242c603" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44535 } } peers { permanent_uuid: "9c938ee6c8584eab855c62e8b0940ed2" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38175 } } peers { permanent_uuid: "210a2731d3a747cf85914d536c79b372" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44741 } }
I20260806 00:58:08.052090 22821 ts_tablet_manager.cc:1428] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260806 00:58:08.052819 22620 heartbeater.cc:503] Master 127.27.77.254:35645 was elected leader, sending a full tablet report...
I20260806 00:58:08.052978 22826 tablet_bootstrap.cc:492] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2: No bootstrap required, opened a new log
I20260806 00:58:08.052979 22823 ts_tablet_manager.cc:1428] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260806 00:58:08.053179 22826 ts_tablet_manager.cc:1397] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:58:08.057447 22826 raft_consensus.cc:348] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169dfec43e184121a54e8f386242c603" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44535 } } peers { permanent_uuid: "9c938ee6c8584eab855c62e8b0940ed2" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38175 } } peers { permanent_uuid: "210a2731d3a747cf85914d536c79b372" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44741 } }
I20260806 00:58:08.057643 22826 raft_consensus.cc:374] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:08.057686 22826 raft_consensus.cc:729] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c938ee6c8584eab855c62e8b0940ed2, State: Initialized, Role: FOLLOWER
I20260806 00:58:08.058099 22826 consensus_queue.cc:260] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169dfec43e184121a54e8f386242c603" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44535 } } peers { permanent_uuid: "9c938ee6c8584eab855c62e8b0940ed2" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38175 } } peers { permanent_uuid: "210a2731d3a747cf85914d536c79b372" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44741 } }
I20260806 00:58:08.058835 22430 heartbeater.cc:503] Master 127.27.77.254:35645 was elected leader, sending a full tablet report...
I20260806 00:58:08.059084 22826 ts_tablet_manager.cc:1428] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260806 00:58:08.230381 22796 tablet.cc:2369] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:58:08.265515 22621 tablet.cc:2369] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:58:08.294049 22831 raft_consensus.cc:482] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:58:08.294315 22831 raft_consensus.cc:504] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169dfec43e184121a54e8f386242c603" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44535 } } peers { permanent_uuid: "9c938ee6c8584eab855c62e8b0940ed2" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38175 } } peers { permanent_uuid: "210a2731d3a747cf85914d536c79b372" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44741 } }
I20260806 00:58:08.295450 22831 leader_election.cc:288] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c938ee6c8584eab855c62e8b0940ed2 (127.27.77.193:38175), 210a2731d3a747cf85914d536c79b372 (127.27.77.194:44741)
I20260806 00:58:08.300591 22816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:08.295961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:58915 (local address 127.27.77.193:38175)
0806 00:58:08.296099 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:08.296103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:08.296185 (+ 82us) server_negotiation.cc:407] Connection header received
0806 00:58:08.296365 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:08.296370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:08.296424 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:08.296525 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:08.296939 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.297689 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.299317 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.299982 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.300108 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:08.300163 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:08.300396 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 00:58:08.300469 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:58:08.300583 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:08.295844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:38175 (local address 127.27.77.195:58915)
0806 00:58:08.296087 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:08.296106 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:08.296238 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:08.296619 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:08.296623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:08.296643 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:08.296800 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:08.296808 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.297828 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.297831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:08.299154 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:08.299166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.300108 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.300111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:08.300254 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:08.300282 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 00:58:08.300364 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 00:58:08.300442 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":35}
I20260806 00:58:08.300846 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:08.296280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.194:44741 (local address 127.27.77.195:41389)
0806 00:58:08.296708 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:08.296722 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:08.296822 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:08.297155 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:08.297157 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:08.297177 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:58:08.297302 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:08.297306 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.298177 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.298181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:08.299385 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:08.299392 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.300368 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.300372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:08.300491 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:08.300511 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:58:08.300631 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 00:58:08.300703 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":23,"thread_start_us":142,"threads_started":1}
I20260806 00:58:08.300839 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:08.296442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.77.195:41389 (local address 127.27.77.194:44741)
0806 00:58:08.296591 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:08.296595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:08.296830 (+ 235us) server_negotiation.cc:407] Connection header received
0806 00:58:08.296931 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:08.296934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:08.296995 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:08.297113 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:58:08.297430 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.298050 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.299576 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.300242 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.300376 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:08.300422 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:08.300653 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 00:58:08.300733 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:58:08.301472 22365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c099210d10f475a911225b28e747b35" candidate_uuid: "169dfec43e184121a54e8f386242c603" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c938ee6c8584eab855c62e8b0940ed2" is_pre_election: true
I20260806 00:58:08.301654 22552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c099210d10f475a911225b28e747b35" candidate_uuid: "169dfec43e184121a54e8f386242c603" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210a2731d3a747cf85914d536c79b372" is_pre_election: true
I20260806 00:58:08.301832 22365 raft_consensus.cc:2454] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 169dfec43e184121a54e8f386242c603 in term 0.
I20260806 00:58:08.301965 22552 raft_consensus.cc:2454] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 169dfec43e184121a54e8f386242c603 in term 0.
W20260806 00:58:08.302433 22432 tablet.cc:2369] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:58:08.302423 22648 leader_election.cc:302] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 169dfec43e184121a54e8f386242c603, 9c938ee6c8584eab855c62e8b0940ed2; no voters:
I20260806 00:58:08.302771 22831 raft_consensus.cc:2781] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:58:08.302876 22831 raft_consensus.cc:482] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:58:08.302922 22831 raft_consensus.cc:3037] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:08.304340 22831 raft_consensus.cc:504] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169dfec43e184121a54e8f386242c603" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44535 } } peers { permanent_uuid: "9c938ee6c8584eab855c62e8b0940ed2" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38175 } } peers { permanent_uuid: "210a2731d3a747cf85914d536c79b372" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44741 } }
I20260806 00:58:08.304714 22831 leader_election.cc:288] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [CANDIDATE]: Term 1 election: Requested vote from peers 9c938ee6c8584eab855c62e8b0940ed2 (127.27.77.193:38175), 210a2731d3a747cf85914d536c79b372 (127.27.77.194:44741)
I20260806 00:58:08.305114 22365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c099210d10f475a911225b28e747b35" candidate_uuid: "169dfec43e184121a54e8f386242c603" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c938ee6c8584eab855c62e8b0940ed2"
I20260806 00:58:08.305176 22552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c099210d10f475a911225b28e747b35" candidate_uuid: "169dfec43e184121a54e8f386242c603" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210a2731d3a747cf85914d536c79b372"
I20260806 00:58:08.305248 22365 raft_consensus.cc:3037] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:08.305318 22552 raft_consensus.cc:3037] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:08.306654 22365 raft_consensus.cc:2454] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 169dfec43e184121a54e8f386242c603 in term 1.
I20260806 00:58:08.306655 22552 raft_consensus.cc:2454] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 169dfec43e184121a54e8f386242c603 in term 1.
I20260806 00:58:08.306988 22648 leader_election.cc:302] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 169dfec43e184121a54e8f386242c603, 210a2731d3a747cf85914d536c79b372; no voters:
I20260806 00:58:08.307165 22831 raft_consensus.cc:2781] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:08.307420 22831 raft_consensus.cc:686] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [term 1 LEADER]: Becoming Leader. State: Replica: 169dfec43e184121a54e8f386242c603, State: Running, Role: LEADER
I20260806 00:58:08.307834 22831 consensus_queue.cc:237] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169dfec43e184121a54e8f386242c603" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44535 } } peers { permanent_uuid: "9c938ee6c8584eab855c62e8b0940ed2" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38175 } } peers { permanent_uuid: "210a2731d3a747cf85914d536c79b372" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44741 } }
I20260806 00:58:08.311041 22153 catalog_manager.cc:5495] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 reported cstate change: term changed from 0 to 1, leader changed from <none> to 169dfec43e184121a54e8f386242c603 (127.27.77.195). New cstate: current_term: 1 leader_uuid: "169dfec43e184121a54e8f386242c603" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "169dfec43e184121a54e8f386242c603" member_type: VOTER last_known_addr { host: "127.27.77.195" port: 44535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c938ee6c8584eab855c62e8b0940ed2" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210a2731d3a747cf85914d536c79b372" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44741 } health_report { overall_health: UNKNOWN } } }
I20260806 00:58:08.392691 22812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:08.388852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48662 (local address 127.27.77.195:44535)
0806 00:58:08.389022 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:08.389026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:08.389041 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:58:08.389129 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:08.389131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:08.389200 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:08.389340 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:58:08.389566 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.390120 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.391047 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:08.391309 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:08.391408 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:08.391827 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:58:08.392284 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:58:08.392393 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:08.392474 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 00:58:08.392529 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:58:08.398988 22552 raft_consensus.cc:1261] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372 [term 1 FOLLOWER]: Refusing update from remote peer 169dfec43e184121a54e8f386242c603: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:58:08.398988 22365 raft_consensus.cc:1261] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2 [term 1 FOLLOWER]: Refusing update from remote peer 169dfec43e184121a54e8f386242c603: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:58:08.399824 22831 consensus_queue.cc:1035] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210a2731d3a747cf85914d536c79b372" member_type: VOTER last_known_addr { host: "127.27.77.194" port: 44741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:58:08.400104 22835 consensus_queue.cc:1035] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c938ee6c8584eab855c62e8b0940ed2" member_type: VOTER last_known_addr { host: "127.27.77.193" port: 38175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:58:08.408583 22841 mvcc.cc:205] Tried to move back new op lower bound from 7315365430873710592 to 7315365430511665152. Current Snapshot: MvccSnapshot[applied={T|T < 7315365430873710592}]
I20260806 00:58:08.417110 22153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:58:08.417311 22153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:58:08.419730 22153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cadf89db55ff4329b03911729d86ea05: Sending DeleteTablet for 3 replicas of tablet 2c099210d10f475a911225b28e747b35
I20260806 00:58:08.420642 22524 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c099210d10f475a911225b28e747b35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:58:08 UTC) from {username='slave'} at 127.0.0.1:54626
I20260806 00:58:08.420652 22706 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c099210d10f475a911225b28e747b35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:58:08 UTC) from {username='slave'} at 127.0.0.1:48654
I20260806 00:58:08.420703 22333 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c099210d10f475a911225b28e747b35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:58:08 UTC) from {username='slave'} at 127.0.0.1:53266
I20260806 00:58:08.421324 22853 tablet_replica.cc:330] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603: stopping tablet replica
I20260806 00:58:08.421396 22854 tablet_replica.cc:330] T 2c099210d10f475a911225b28e747b35 P 9c938ee6c8584eab855c62e8b0940ed2: stopping tablet replica
I20260806 00:58:08.421640 22853 raft_consensus.cc:2229] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:58:08.421993 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 22243
I20260806 00:58:08.422010 22853 raft_consensus.cc:2258] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:58:08.422374 22852 tablet_replica.cc:330] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372: stopping tablet replica
I20260806 00:58:08.422645 22852 raft_consensus.cc:2229] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:58:08.422999 22852 raft_consensus.cc:2258] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:58:08.426651 22852 ts_tablet_manager.cc:1905] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:58:08.427145 22853 ts_tablet_manager.cc:1905] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:58:08.429814 22852 ts_tablet_manager.cc:1918] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:58:08.429814 22853 ts_tablet_manager.cc:1918] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:58:08.429919 22853 log.cc:1196] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-2/wal/wals/2c099210d10f475a911225b28e747b35
I20260806 00:58:08.430063 22852 log.cc:1196] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372: Deleting WAL directory at /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977619051187-27959-0/minicluster-data/ts-1/wal/wals/2c099210d10f475a911225b28e747b35
I20260806 00:58:08.430346 22853 ts_tablet_manager.cc:1939] T 2c099210d10f475a911225b28e747b35 P 169dfec43e184121a54e8f386242c603: Deleting consensus metadata
I20260806 00:58:08.430549 22852 ts_tablet_manager.cc:1939] T 2c099210d10f475a911225b28e747b35 P 210a2731d3a747cf85914d536c79b372: Deleting consensus metadata
I20260806 00:58:08.431917 22137 catalog_manager.cc:4841] TS 210a2731d3a747cf85914d536c79b372 (127.27.77.194:44741): tablet 2c099210d10f475a911225b28e747b35 (table test-table [id=4dc4b94042764077b8a33a1d8e56ab84]) successfully deleted
I20260806 00:58:08.432555 22134 catalog_manager.cc:4841] TS 169dfec43e184121a54e8f386242c603 (127.27.77.195:44535): tablet 2c099210d10f475a911225b28e747b35 (table test-table [id=4dc4b94042764077b8a33a1d8e56ab84]) successfully deleted
W20260806 00:58:08.450263 22137 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.77.193:38175 (error 108)
I20260806 00:58:08.450387 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 22435
I20260806 00:58:08.456638 22811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:08.456302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.193:38175 (local address 127.0.0.1:53276)
0806 00:58:08.456453 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:08.456546 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.193:38175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 00:58:08.456955 22137 catalog_manager.cc:4584] TS 9c938ee6c8584eab855c62e8b0940ed2 (127.27.77.193:38175): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c099210d10f475a911225b28e747b35: Network error: Client connection negotiation failed: client connection to 127.27.77.193:38175: connect: Connection refused (error 111)
I20260806 00:58:08.462296 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 22623
I20260806 00:58:08.473531 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 22110
2026-08-06T00:58:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1409 ms)
[----------] 6 tests from EncryptionPolicyTest (12085 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:08 dist-test-slave-sb2z krb5kdc[22860](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:08 dist-test-slave-sb2z krb5kdc[22860](info): set up 2 sockets
Aug 06 00:58:08 dist-test-slave-sb2z krb5kdc[22860](info): commencing operation
krb5kdc: starting...
W20260806 00:58:09.710398 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.008s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:09 dist-test-slave-sb2z krb5kdc[22860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:58:09.852797 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42387
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:09.972702 22873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:09.973136 22873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:09.973260 22873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:09.977415 22873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:09.977536 22873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:09.977577 22873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:09.977618 22873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:09.977654 22873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:09.983681 22873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42387
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:09.984926 22873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:09.986016 22873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:09.994992 22879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:09.995144 22873 server_base.cc:1034] running on GCE node
W20260806 00:58:09.994953 22878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:09.994972 22881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:09.996001 22873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:09.997090 22873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:09.998275 22873 hybrid_clock.cc:648] HybridClock initialized: now 1785977889998254 us; error 39 us; skew 500 ppm
Aug 06 00:58:09 dist-test-slave-sb2z krb5kdc[22860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:10.001549 22873 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:58:10.002537 22873 webserver.cc:460] Webserver started at http://127.0.0.1:41965/ using document root <none> and password file <none>
I20260806 00:58:10.003134 22873 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:10.003232 22873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:10.003484 22873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:10.005671 22873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "245ccb5243ab429ea2b15c2eb0bbca2a"
format_stamp: "Formatted at 2026-08-06 00:58:10 on dist-test-slave-sb2z"
server_key: "32b1421f09649f088a898328600d6c5b"
server_key_iv: "3b7dc9c2868808772ab210c83d9a72b1"
server_key_version: "encryptionkey@0"
I20260806 00:58:10.006275 22873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "245ccb5243ab429ea2b15c2eb0bbca2a"
format_stamp: "Formatted at 2026-08-06 00:58:10 on dist-test-slave-sb2z"
server_key: "32b1421f09649f088a898328600d6c5b"
server_key_iv: "3b7dc9c2868808772ab210c83d9a72b1"
server_key_version: "encryptionkey@0"
I20260806 00:58:10.010432 22873 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 00:58:10.013696 22893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:10.015129 22873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 00:58:10.015298 22873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "245ccb5243ab429ea2b15c2eb0bbca2a"
format_stamp: "Formatted at 2026-08-06 00:58:10 on dist-test-slave-sb2z"
server_key: "32b1421f09649f088a898328600d6c5b"
server_key_iv: "3b7dc9c2868808772ab210c83d9a72b1"
server_key_version: "encryptionkey@0"
I20260806 00:58:10.015434 22873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:10.036109 22873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:10.039947 22873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:10.040221 22873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:10.048418 22873 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42387
I20260806 00:58:10.048441 22968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42387 every 8 connection(s)
I20260806 00:58:10.049465 22873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:10.051828 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 22873
I20260806 00:58:10.051988 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:10.052305 27959 external_mini_cluster.cc:1428] Setting key 189b6835234eb522a0a3a9024a274671
I20260806 00:58:10.052798 22969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:10.059661 22969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a: Bootstrap starting.
Aug 06 00:58:10 dist-test-slave-sb2z krb5kdc[22860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 00:58:10.062561 22969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:10.063647 22969 log.cc:824] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:10.065805 22969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a: No bootstrap required, opened a new log
I20260806 00:58:10.067781 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:10.053862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53687 (local address 127.0.0.1:42387)
0806 00:58:10.054358 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:10.054373 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:10.054411 (+ 38us) server_negotiation.cc:407] Connection header received
0806 00:58:10.055110 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:10.055133 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:10.055468 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:10.055864 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:10.056708 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:10.057776 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:10.058671 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:10.059161 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:10.059349 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:10.062446 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:10.062452 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:10.062466 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:10.062496 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:10.064404 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:10.064959 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:10.064965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:10.064970 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:10.065135 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:10.065497 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:10.065502 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:10.065509 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:10.065956 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:10.066188 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:10.066524 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 00:58:10.066765 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":165,"threads_started":1}
I20260806 00:58:10.069398 22969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245ccb5243ab429ea2b15c2eb0bbca2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42387 } }
I20260806 00:58:10.069698 22969 raft_consensus.cc:374] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:10.069782 22969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245ccb5243ab429ea2b15c2eb0bbca2a, State: Initialized, Role: FOLLOWER
I20260806 00:58:10.070276 22969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245ccb5243ab429ea2b15c2eb0bbca2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42387 } }
I20260806 00:58:10.070462 22969 raft_consensus.cc:388] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:10.070581 22969 raft_consensus.cc:482] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:10.070695 22969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:10.071187 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:10.072093 22969 raft_consensus.cc:504] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245ccb5243ab429ea2b15c2eb0bbca2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42387 } }
I20260806 00:58:10.072522 22969 leader_election.cc:302] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 245ccb5243ab429ea2b15c2eb0bbca2a; no voters:
I20260806 00:58:10.072970 22969 leader_election.cc:288] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:10.073117 22981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:10.073407 22981 raft_consensus.cc:686] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [term 1 LEADER]: Becoming Leader. State: Replica: 245ccb5243ab429ea2b15c2eb0bbca2a, State: Running, Role: LEADER
I20260806 00:58:10.074322 22981 consensus_queue.cc:237] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245ccb5243ab429ea2b15c2eb0bbca2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42387 } }
I20260806 00:58:10.074462 22969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:10.076247 22985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "245ccb5243ab429ea2b15c2eb0bbca2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245ccb5243ab429ea2b15c2eb0bbca2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42387 } } }
I20260806 00:58:10.076411 22985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:10.076767 22983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 245ccb5243ab429ea2b15c2eb0bbca2a. Latest consensus state: current_term: 1 leader_uuid: "245ccb5243ab429ea2b15c2eb0bbca2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245ccb5243ab429ea2b15c2eb0bbca2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42387 } } }
I20260806 00:58:10.076897 22983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a [sys.catalog]: This master's current role is: LEADER
W20260806 00:58:10.080658 23001 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:58:10.080755 23001 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:58:10.080838 22997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:10.081911 22997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:10.083825 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:10.073811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51613 (local address 127.0.0.1:42387)
0806 00:58:10.073957 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:10.073962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:10.074196 (+ 234us) server_negotiation.cc:407] Connection header received
0806 00:58:10.074363 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:10.074369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:10.074420 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:10.074501 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:10.075331 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:10.076283 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:10.077611 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:10.077939 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:10.078063 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:10.079866 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:10.079870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:10.079873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:10.079909 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:10.081666 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:10.082317 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:10.082321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:10.082323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:10.082418 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:10.082800 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:10.082804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:10.082806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:10.083011 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:10.083184 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:10.083498 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 00:58:10.083631 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:58:10.088845 22997 catalog_manager.cc:1347] Generated new cluster ID: 53eb9044f2a64a6f91b43bb728747577
I20260806 00:58:10.088935 22997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:10.151099 22997 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:10.152349 22997 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:10.168833 22997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 245ccb5243ab429ea2b15c2eb0bbca2a: Generated new TSK 0
I20260806 00:58:10.169808 22997 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:10.179078 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 22873
W20260806 00:58:10.196134 22857 connection.cc:535] client connection to 127.0.0.1:42387 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42387): Connection reset by peer (error 104)
2026-08-06T00:58:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1710 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:10 dist-test-slave-sb2z krb5kdc[23012](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:10 dist-test-slave-sb2z krb5kdc[23012](info): set up 2 sockets
Aug 06 00:58:10 dist-test-slave-sb2z krb5kdc[23012](info): commencing operation
krb5kdc: starting...
W20260806 00:58:11.392248 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:11 dist-test-slave-sb2z krb5kdc[23012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:58:11.535657 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43873
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:11.656688 23026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:11.657063 23026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:11.657179 23026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:11.661770 23026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:11.661886 23026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:11.661931 23026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:11.661973 23026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:11.662007 23026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:11.668001 23026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43873
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:11.669207 23026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:11.670310 23026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:11.679411 23033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:11.679347 23034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:11.679593 23026 server_base.cc:1034] running on GCE node
W20260806 00:58:11.679348 23038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:11.680255 23026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:11.681545 23026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:11.682734 23026 hybrid_clock.cc:648] HybridClock initialized: now 1785977891682704 us; error 41 us; skew 500 ppm
Aug 06 00:58:11 dist-test-slave-sb2z krb5kdc[23012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977891, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:11.686156 23026 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:58:11.687094 23026 webserver.cc:460] Webserver started at http://127.0.0.1:32859/ using document root <none> and password file <none>
I20260806 00:58:11.687748 23026 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:11.687844 23026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:11.688089 23026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:11.690223 23026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "147f62a8640b40aebfab85585cee74f3"
format_stamp: "Formatted at 2026-08-06 00:58:11 on dist-test-slave-sb2z"
server_key: "527b2b995ae26245dcb85681e57ddf19"
server_key_iv: "9bfd5becefab9bcfd6eb6f024eff5ff1"
server_key_version: "encryptionkey@0"
I20260806 00:58:11.690820 23026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "147f62a8640b40aebfab85585cee74f3"
format_stamp: "Formatted at 2026-08-06 00:58:11 on dist-test-slave-sb2z"
server_key: "527b2b995ae26245dcb85681e57ddf19"
server_key_iv: "9bfd5becefab9bcfd6eb6f024eff5ff1"
server_key_version: "encryptionkey@0"
I20260806 00:58:11.694784 23026 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:58:11.698158 23048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:11.700084 23026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:11.700245 23026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "147f62a8640b40aebfab85585cee74f3"
format_stamp: "Formatted at 2026-08-06 00:58:11 on dist-test-slave-sb2z"
server_key: "527b2b995ae26245dcb85681e57ddf19"
server_key_iv: "9bfd5becefab9bcfd6eb6f024eff5ff1"
server_key_version: "encryptionkey@0"
I20260806 00:58:11.700376 23026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:11.736714 23026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:11.740579 23026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:11.740833 23026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:11.748739 23026 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43873
I20260806 00:58:11.748745 23120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43873 every 8 connection(s)
I20260806 00:58:11.749871 23026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:11.753320 23121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:11.757634 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 23026
I20260806 00:58:11.757833 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:11.758213 27959 external_mini_cluster.cc:1428] Setting key 785101b370c8486ff6927cabcf57f533
I20260806 00:58:11.759846 23121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3: Bootstrap starting.
I20260806 00:58:11.762825 23121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:11.763816 23121 log.cc:824] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:11.766176 23121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3: No bootstrap required, opened a new log
I20260806 00:58:11.769767 23124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:11.759848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57943 (local address 127.0.0.1:43873)
0806 00:58:11.760366 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:11.760385 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:11.760429 (+ 44us) server_negotiation.cc:407] Connection header received
0806 00:58:11.761380 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:11.761404 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:11.761857 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:11.762249 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:11.762694 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:11.764299 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:11.765485 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:11.766160 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:11.766351 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:11.767413 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:11.767420 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:11.767441 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:11.768008 (+ 567us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:11.768082 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:11.768116 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:11.768268 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:11.768507 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 00:58:11.768657 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":138,"threads_started":1}
I20260806 00:58:11.770939 23121 raft_consensus.cc:348] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "147f62a8640b40aebfab85585cee74f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43873 } }
I20260806 00:58:11.771219 23121 raft_consensus.cc:374] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:11.771286 23121 raft_consensus.cc:729] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147f62a8640b40aebfab85585cee74f3, State: Initialized, Role: FOLLOWER
I20260806 00:58:11.771795 23121 consensus_queue.cc:260] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "147f62a8640b40aebfab85585cee74f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43873 } }
I20260806 00:58:11.771989 23121 raft_consensus.cc:388] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:11.772061 23121 raft_consensus.cc:482] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:11.772152 23121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:11.773331 23121 raft_consensus.cc:504] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "147f62a8640b40aebfab85585cee74f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43873 } }
I20260806 00:58:11.773357 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:11.773705 23121 leader_election.cc:302] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 147f62a8640b40aebfab85585cee74f3; no voters:
I20260806 00:58:11.774097 23121 leader_election.cc:288] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:11.774194 23126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:11.774386 23126 raft_consensus.cc:686] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [term 1 LEADER]: Becoming Leader. State: Replica: 147f62a8640b40aebfab85585cee74f3, State: Running, Role: LEADER
I20260806 00:58:11.774712 23126 consensus_queue.cc:237] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "147f62a8640b40aebfab85585cee74f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43873 } }
I20260806 00:58:11.775499 23121 sys_catalog.cc:564] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:11.776681 23127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "147f62a8640b40aebfab85585cee74f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "147f62a8640b40aebfab85585cee74f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43873 } } }
I20260806 00:58:11.776830 23127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:11.777204 23128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 147f62a8640b40aebfab85585cee74f3. Latest consensus state: current_term: 1 leader_uuid: "147f62a8640b40aebfab85585cee74f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "147f62a8640b40aebfab85585cee74f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43873 } } }
I20260806 00:58:11.777312 23128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:11.778229 23145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:11.780982 23124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:11.776021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54563 (local address 127.0.0.1:43873)
0806 00:58:11.776135 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:11.776141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:11.776413 (+ 272us) server_negotiation.cc:407] Connection header received
0806 00:58:11.776661 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:11.776664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:11.776721 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:11.776776 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:11.777446 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:11.778209 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:11.779196 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:11.779440 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:11.779569 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:11.780449 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:11.780452 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:11.780455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:11.780469 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:11.780503 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:11.780514 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:11.780620 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:11.780774 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 00:58:11.780812 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:58:11.781607 23145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:11.787309 23145 catalog_manager.cc:1347] Generated new cluster ID: fae846cef56c4fb6b25cb311198c378d
I20260806 00:58:11.787386 23145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:11.797986 23145 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:11.799023 23145 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:11.804445 23145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 147f62a8640b40aebfab85585cee74f3: Generated new TSK 0
I20260806 00:58:11.805202 23145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:11.815032 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 23026
W20260806 00:58:11.828465 23008 connection.cc:535] client connection to 127.0.0.1:43873 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43873): Connection reset by peer (error 104)
2026-08-06T00:58:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1632 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:11 dist-test-slave-sb2z krb5kdc[23163](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:11 dist-test-slave-sb2z krb5kdc[23163](info): set up 2 sockets
Aug 06 00:58:11 dist-test-slave-sb2z krb5kdc[23163](info): commencing operation
krb5kdc: starting...
W20260806 00:58:13.064229 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:13 dist-test-slave-sb2z krb5kdc[23163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:58:13.207958 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44269
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:13.328488 23182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:13.328840 23182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:13.328924 23182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:13.333115 23182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:13.333225 23182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:13.333266 23182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:13.333304 23182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:13.333338 23182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:13.339354 23182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44269
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:13.340642 23182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:13.341792 23182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:13.351305 23189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:13.351317 23188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:13.351423 23182 server_base.cc:1034] running on GCE node
W20260806 00:58:13.351323 23192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:13.352247 23182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:13.353446 23182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:13.354630 23182 hybrid_clock.cc:648] HybridClock initialized: now 1785977893354598 us; error 47 us; skew 500 ppm
Aug 06 00:58:13 dist-test-slave-sb2z krb5kdc[23163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977893, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:13.361514 23182 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:58:13.362648 23182 webserver.cc:460] Webserver started at http://127.0.0.1:40177/ using document root <none> and password file <none>
I20260806 00:58:13.363209 23182 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:13.363272 23182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:13.363442 23182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:13.365700 23182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "b4530b2ea00a4c4ea7aa36d882cbd8b6"
format_stamp: "Formatted at 2026-08-06 00:58:13 on dist-test-slave-sb2z"
server_key: "e82a47f2ce58250882d2a8f1f8694b3c"
server_key_iv: "98cb9a9a5ae49c418b14e4abe6d42c2e"
server_key_version: "encryptionkey@0"
I20260806 00:58:13.366248 23182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "b4530b2ea00a4c4ea7aa36d882cbd8b6"
format_stamp: "Formatted at 2026-08-06 00:58:13 on dist-test-slave-sb2z"
server_key: "e82a47f2ce58250882d2a8f1f8694b3c"
server_key_iv: "98cb9a9a5ae49c418b14e4abe6d42c2e"
server_key_version: "encryptionkey@0"
I20260806 00:58:13.370874 23182 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260806 00:58:13.373596 23199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:13.375000 23182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 00:58:13.375154 23182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "b4530b2ea00a4c4ea7aa36d882cbd8b6"
format_stamp: "Formatted at 2026-08-06 00:58:13 on dist-test-slave-sb2z"
server_key: "e82a47f2ce58250882d2a8f1f8694b3c"
server_key_iv: "98cb9a9a5ae49c418b14e4abe6d42c2e"
server_key_version: "encryptionkey@0"
I20260806 00:58:13.375269 23182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:13.380172 23182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:13.383514 23182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:13.383831 23182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:13.391486 23182 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44269
I20260806 00:58:13.391494 23275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44269 every 8 connection(s)
I20260806 00:58:13.392531 23182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:13.395360 23277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:13.396160 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 23182
I20260806 00:58:13.396314 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:13.396653 27959 external_mini_cluster.cc:1428] Setting key c2006dd8e4720f22a8f882dbd2436116
I20260806 00:58:13.402632 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:13.398017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57763 (local address 127.0.0.1:44269)
0806 00:58:13.398483 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:13.398505 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:13.398553 (+ 48us) server_negotiation.cc:407] Connection header received
0806 00:58:13.399417 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:13.399441 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:13.399835 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:13.400213 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:13.401130 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:13.401136 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:13.401152 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:13.401165 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:13.401235 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:13.401265 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:13.401421 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:13.401563 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 00:58:13.401721 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":115,"threads_started":1}
I20260806 00:58:13.402659 23277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6: Bootstrap starting.
I20260806 00:58:13.405361 23277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:13.405915 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:13.406275 23277 log.cc:824] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:13.408809 23277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6: No bootstrap required, opened a new log
I20260806 00:58:13.410225 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:13.408333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57925 (local address 127.0.0.1:44269)
0806 00:58:13.408472 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:13.408476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:13.408762 (+ 286us) server_negotiation.cc:407] Connection header received
0806 00:58:13.408949 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:13.408955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:13.408996 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:13.409070 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:13.409819 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:13.409822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:13.409823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:13.409834 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:13.409860 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:13.409870 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:13.409945 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:13.410098 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 00:58:13.410133 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:58:13.414111 23277 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4530b2ea00a4c4ea7aa36d882cbd8b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44269 } }
I20260806 00:58:13.414369 23277 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:13.414408 23277 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4530b2ea00a4c4ea7aa36d882cbd8b6, State: Initialized, Role: FOLLOWER
I20260806 00:58:13.414810 23277 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4530b2ea00a4c4ea7aa36d882cbd8b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44269 } }
I20260806 00:58:13.414930 23277 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:13.414975 23277 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:13.415055 23277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:13.416219 23277 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4530b2ea00a4c4ea7aa36d882cbd8b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44269 } }
I20260806 00:58:13.416551 23277 leader_election.cc:302] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4530b2ea00a4c4ea7aa36d882cbd8b6; no voters:
I20260806 00:58:13.416922 23277 leader_election.cc:288] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:13.417022 23293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:13.417315 23293 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [term 1 LEADER]: Becoming Leader. State: Replica: b4530b2ea00a4c4ea7aa36d882cbd8b6, State: Running, Role: LEADER
I20260806 00:58:13.417713 23293 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4530b2ea00a4c4ea7aa36d882cbd8b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44269 } }
I20260806 00:58:13.418404 23277 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:13.419718 23295 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4530b2ea00a4c4ea7aa36d882cbd8b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4530b2ea00a4c4ea7aa36d882cbd8b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44269 } } }
I20260806 00:58:13.419718 23296 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4530b2ea00a4c4ea7aa36d882cbd8b6. Latest consensus state: current_term: 1 leader_uuid: "b4530b2ea00a4c4ea7aa36d882cbd8b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4530b2ea00a4c4ea7aa36d882cbd8b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44269 } } }
I20260806 00:58:13.419909 23295 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:13.419974 23296 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:13.420737 23307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:13.424183 23307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:13.430598 23307 catalog_manager.cc:1347] Generated new cluster ID: 51aca815091a49deab087ad4b62a6dd3
I20260806 00:58:13.430688 23307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:13.438845 23307 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:13.439188 23307 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:13.447206 23307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4530b2ea00a4c4ea7aa36d882cbd8b6: Generated new TSK 0
I20260806 00:58:13.448019 23307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:13.457563 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 23182
2026-08-06T00:58:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1643 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:13 dist-test-slave-sb2z krb5kdc[23325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:13 dist-test-slave-sb2z krb5kdc[23325](info): set up 2 sockets
Aug 06 00:58:13 dist-test-slave-sb2z krb5kdc[23325](info): commencing operation
krb5kdc: starting...
W20260806 00:58:14.692485 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:14 dist-test-slave-sb2z krb5kdc[23325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:58:14.830452 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44017
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:14.948764 23345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:14.949055 23345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:14.949100 23345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:14.953230 23345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:14.953294 23345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:14.953313 23345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:14.953330 23345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:14.953346 23345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:14.958940 23345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:14.960160 23345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:14.961272 23345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:14.970463 23359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:14.970511 23356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:14.970538 23345 server_base.cc:1034] running on GCE node
W20260806 00:58:14.970474 23357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:14.971395 23345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:14.972764 23345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:14.973973 23345 hybrid_clock.cc:648] HybridClock initialized: now 1785977894973934 us; error 65 us; skew 500 ppm
Aug 06 00:58:14 dist-test-slave-sb2z krb5kdc[23325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977894, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:14.977191 23345 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:58:14.978193 23345 webserver.cc:460] Webserver started at http://127.0.0.1:33035/ using document root <none> and password file <none>
I20260806 00:58:14.978734 23345 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:14.978793 23345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:14.978964 23345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:14.981122 23345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "eb5d9cae9ed04bb288202f9c553250ee"
format_stamp: "Formatted at 2026-08-06 00:58:14 on dist-test-slave-sb2z"
server_key: "05189523b0e42d4309870b06f91fbe3c"
server_key_iv: "a1890255ee7f4377d06ac2e09f9ebae3"
server_key_version: "encryptionkey@0"
I20260806 00:58:14.981664 23345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "eb5d9cae9ed04bb288202f9c553250ee"
format_stamp: "Formatted at 2026-08-06 00:58:14 on dist-test-slave-sb2z"
server_key: "05189523b0e42d4309870b06f91fbe3c"
server_key_iv: "a1890255ee7f4377d06ac2e09f9ebae3"
server_key_version: "encryptionkey@0"
I20260806 00:58:14.985667 23345 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:58:14.988329 23367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:14.989739 23345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:14.989851 23345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "eb5d9cae9ed04bb288202f9c553250ee"
format_stamp: "Formatted at 2026-08-06 00:58:14 on dist-test-slave-sb2z"
server_key: "05189523b0e42d4309870b06f91fbe3c"
server_key_iv: "a1890255ee7f4377d06ac2e09f9ebae3"
server_key_version: "encryptionkey@0"
I20260806 00:58:14.989941 23345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:15.001571 23345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:15.004801 23345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:15.004968 23345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:15.012295 23345 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44017
I20260806 00:58:15.012322 23440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44017 every 8 connection(s)
I20260806 00:58:15.013286 23345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:15.016273 23441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:15.018833 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 23345
I20260806 00:58:15.018953 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:15.019357 27959 external_mini_cluster.cc:1428] Setting key 2f32bf099ace076923ad212cd3359416
I20260806 00:58:15.022907 23441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee: Bootstrap starting.
I20260806 00:58:15.025393 23441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:15.026222 23441 log.cc:824] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:15.028344 23441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee: No bootstrap required, opened a new log
Aug 06 00:58:15 dist-test-slave-sb2z krb5kdc[23325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 00:58:15.031566 23441 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5d9cae9ed04bb288202f9c553250ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44017 } }
I20260806 00:58:15.031854 23441 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:15.031949 23441 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5d9cae9ed04bb288202f9c553250ee, State: Initialized, Role: FOLLOWER
I20260806 00:58:15.032461 23441 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5d9cae9ed04bb288202f9c553250ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44017 } }
I20260806 00:58:15.032622 23441 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:15.032727 23441 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:15.032859 23441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:15.034042 23441 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5d9cae9ed04bb288202f9c553250ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44017 } }
I20260806 00:58:15.034441 23441 leader_election.cc:302] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb5d9cae9ed04bb288202f9c553250ee; no voters:
I20260806 00:58:15.034771 23441 leader_election.cc:288] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:15.034889 23448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:15.035125 23448 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [term 1 LEADER]: Becoming Leader. State: Replica: eb5d9cae9ed04bb288202f9c553250ee, State: Running, Role: LEADER
I20260806 00:58:15.035487 23448 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5d9cae9ed04bb288202f9c553250ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44017 } }
I20260806 00:58:15.035724 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:15.021020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43987 (local address 127.0.0.1:44017)
0806 00:58:15.021575 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:15.021592 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:15.021634 (+ 42us) server_negotiation.cc:407] Connection header received
0806 00:58:15.022353 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:15.022373 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:15.022741 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:15.023148 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:15.024073 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:15.025113 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:15.026095 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:15.026647 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:15.026836 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:15.029522 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:15.029528 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:15.029543 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:15.029574 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:15.032158 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:15.032634 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:15.032641 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:15.032649 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:15.032810 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:15.033153 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:15.033156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:15.033157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:15.033560 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:15.033738 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:15.034187 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 00:58:15.034586 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":150,"threads_started":1}
I20260806 00:58:15.036242 23441 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:15.037248 23449 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb5d9cae9ed04bb288202f9c553250ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5d9cae9ed04bb288202f9c553250ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44017 } } }
I20260806 00:58:15.037370 23449 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:15.037602 23449 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb5d9cae9ed04bb288202f9c553250ee. Latest consensus state: current_term: 1 leader_uuid: "eb5d9cae9ed04bb288202f9c553250ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5d9cae9ed04bb288202f9c553250ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44017 } } }
I20260806 00:58:15.037674 23449 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:15.038055 23458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:15.039304 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:15.042258 23458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:15.048820 23458 catalog_manager.cc:1347] Generated new cluster ID: 875905f12af54dd6bb4a89193995336a
I20260806 00:58:15.048929 23458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:15.051350 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:15.041687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54975 (local address 127.0.0.1:44017)
0806 00:58:15.041836 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:15.041841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:15.042071 (+ 230us) server_negotiation.cc:407] Connection header received
0806 00:58:15.042180 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:15.042182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:15.042233 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:15.042333 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:15.043213 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:15.044026 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:15.045274 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:15.045604 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:15.045713 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:15.047488 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:15.047492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:15.047495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:15.047573 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:15.049416 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:15.049969 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:15.049973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:15.049975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:15.050076 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:15.050377 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:15.050380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:15.050382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:15.050633 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:15.050774 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:15.051014 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 00:58:15.051166 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:58:15.075764 23458 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:15.076730 23458 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:15.085480 23458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb5d9cae9ed04bb288202f9c553250ee: Generated new TSK 0
I20260806 00:58:15.086211 23458 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:15.094710 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 23345
2026-08-06T00:58:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1635 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:15 dist-test-slave-sb2z krb5kdc[23483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:15 dist-test-slave-sb2z krb5kdc[23483](info): set up 2 sockets
Aug 06 00:58:15 dist-test-slave-sb2z krb5kdc[23483](info): commencing operation
krb5kdc: starting...
W20260806 00:58:16.330152 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:16 dist-test-slave-sb2z krb5kdc[23483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:58:16.472093 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42297
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:16.591351 23502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:16.591738 23502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:16.591830 23502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:16.595885 23502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:16.595999 23502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:16.596037 23502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:16.596074 23502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:16.596109 23502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:16.601701 23502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:16.602876 23502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:16.604010 23502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:16.612829 23512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:16.613058 23502 server_base.cc:1034] running on GCE node
W20260806 00:58:16.612843 23513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:16.612869 23515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:16.613799 23502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:16.614945 23502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:16.616153 23502 hybrid_clock.cc:648] HybridClock initialized: now 1785977896616131 us; error 32 us; skew 500 ppm
Aug 06 00:58:16 dist-test-slave-sb2z krb5kdc[23483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977896, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:16.623296 23502 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:58:16.624441 23502 webserver.cc:460] Webserver started at http://127.0.0.1:43573/ using document root <none> and password file <none>
I20260806 00:58:16.625059 23502 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:16.625160 23502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:16.625406 23502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:16.627517 23502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "e85d28c43caa4937b11b2d5ba2721884"
format_stamp: "Formatted at 2026-08-06 00:58:16 on dist-test-slave-sb2z"
server_key: "c49fd2e78ffd8dc80ddb3b9cb9cc4b7c"
server_key_iv: "77ecdb6d726b99fe33187b82e92fcc7b"
server_key_version: "encryptionkey@0"
I20260806 00:58:16.628218 23502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "e85d28c43caa4937b11b2d5ba2721884"
format_stamp: "Formatted at 2026-08-06 00:58:16 on dist-test-slave-sb2z"
server_key: "c49fd2e78ffd8dc80ddb3b9cb9cc4b7c"
server_key_iv: "77ecdb6d726b99fe33187b82e92fcc7b"
server_key_version: "encryptionkey@0"
I20260806 00:58:16.632311 23502 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:58:16.634930 23525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:16.636371 23502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:58:16.636521 23502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "e85d28c43caa4937b11b2d5ba2721884"
format_stamp: "Formatted at 2026-08-06 00:58:16 on dist-test-slave-sb2z"
server_key: "c49fd2e78ffd8dc80ddb3b9cb9cc4b7c"
server_key_iv: "77ecdb6d726b99fe33187b82e92fcc7b"
server_key_version: "encryptionkey@0"
I20260806 00:58:16.636648 23502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:16.654193 23502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:16.657482 23502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:16.657717 23502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:16.665699 23502 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42297
I20260806 00:58:16.665720 23591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42297 every 8 connection(s)
I20260806 00:58:16.666751 23502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:16.669659 23593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:16.670893 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 23502
I20260806 00:58:16.671011 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:16.671391 27959 external_mini_cluster.cc:1428] Setting key eeb5f8cda5d7a7e227f111b693e66156
I20260806 00:58:16.676680 23593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884: Bootstrap starting.
I20260806 00:58:16.679503 23593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:16.680490 23593 log.cc:824] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:16.681936 23598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:16.673000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38091 (local address 127.0.0.1:42297)
0806 00:58:16.673460 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:16.673479 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:16.673523 (+ 44us) server_negotiation.cc:407] Connection header received
0806 00:58:16.674350 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:16.674374 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:16.674816 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:16.675199 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:16.675662 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:16.677074 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:16.678059 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:16.678740 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:16.678904 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:16.679871 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:16.679879 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:16.679897 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:16.680411 (+ 514us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:16.680479 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:16.680512 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:16.680622 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:16.680804 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 00:58:16.681053 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":118,"threads_started":1}
I20260806 00:58:16.682767 23593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884: No bootstrap required, opened a new log
I20260806 00:58:16.685002 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:16.686957 23593 raft_consensus.cc:348] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85d28c43caa4937b11b2d5ba2721884" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42297 } }
I20260806 00:58:16.687261 23593 raft_consensus.cc:374] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:16.687366 23593 raft_consensus.cc:729] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e85d28c43caa4937b11b2d5ba2721884, State: Initialized, Role: FOLLOWER
I20260806 00:58:16.687899 23593 consensus_queue.cc:260] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85d28c43caa4937b11b2d5ba2721884" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42297 } }
I20260806 00:58:16.688057 23593 raft_consensus.cc:388] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:16.688145 23593 raft_consensus.cc:482] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:16.688256 23593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:16.689251 23593 raft_consensus.cc:504] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85d28c43caa4937b11b2d5ba2721884" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42297 } }
I20260806 00:58:16.689648 23593 leader_election.cc:302] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e85d28c43caa4937b11b2d5ba2721884; no voters:
I20260806 00:58:16.689999 23593 leader_election.cc:288] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:16.690111 23606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:16.690371 23606 raft_consensus.cc:686] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [term 1 LEADER]: Becoming Leader. State: Replica: e85d28c43caa4937b11b2d5ba2721884, State: Running, Role: LEADER
I20260806 00:58:16.690793 23606 consensus_queue.cc:237] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85d28c43caa4937b11b2d5ba2721884" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42297 } }
I20260806 00:58:16.691483 23593 sys_catalog.cc:564] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:16.691843 23598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:16.687197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38015 (local address 127.0.0.1:42297)
0806 00:58:16.687387 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:16.687392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:16.687634 (+ 242us) server_negotiation.cc:407] Connection header received
0806 00:58:16.687776 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:16.687779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:16.687828 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:16.687914 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:16.688175 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:16.688698 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:16.689635 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:16.690061 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:16.690848 (+ 787us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:16.691208 (+ 360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:16.691212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:16.691215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:16.691230 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:16.691262 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:16.691386 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:16.691469 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:16.691656 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 00:58:16.691714 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 00:58:16.692730 23607 sys_catalog.cc:455] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e85d28c43caa4937b11b2d5ba2721884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85d28c43caa4937b11b2d5ba2721884" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42297 } } }
I20260806 00:58:16.692871 23607 sys_catalog.cc:458] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:16.693159 23609 sys_catalog.cc:455] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e85d28c43caa4937b11b2d5ba2721884. Latest consensus state: current_term: 1 leader_uuid: "e85d28c43caa4937b11b2d5ba2721884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85d28c43caa4937b11b2d5ba2721884" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42297 } } }
I20260806 00:58:16.693300 23609 sys_catalog.cc:458] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:16.693459 23615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:16.696934 23615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:16.703240 23615 catalog_manager.cc:1347] Generated new cluster ID: 9005e82ab964450ca034e81011bada0b
I20260806 00:58:16.703329 23615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:16.721630 23615 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:16.723027 23615 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:16.733798 23615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e85d28c43caa4937b11b2d5ba2721884: Generated new TSK 0
I20260806 00:58:16.734723 23615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:16.739702 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 23502
2026-08-06T00:58:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1642 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:16 dist-test-slave-sb2z krb5kdc[23635](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:16 dist-test-slave-sb2z krb5kdc[23635](info): set up 2 sockets
Aug 06 00:58:16 dist-test-slave-sb2z krb5kdc[23635](info): commencing operation
krb5kdc: starting...
W20260806 00:58:17.941527 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:18 dist-test-slave-sb2z krb5kdc[23635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:58:18.082659 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43489
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:18.197824 23652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:18.198103 23652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:18.198148 23652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:18.202240 23652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:18.202309 23652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:18.202329 23652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:18.202345 23652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:18.202360 23652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:18.207955 23652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:18.209079 23652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:18.210144 23652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:18.219508 23660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:18.219508 23657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:18.219702 23652 server_base.cc:1034] running on GCE node
W20260806 00:58:18.219944 23662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:18.220567 23652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:18.221712 23652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:18.222913 23652 hybrid_clock.cc:648] HybridClock initialized: now 1785977898222859 us; error 66 us; skew 500 ppm
Aug 06 00:58:18 dist-test-slave-sb2z krb5kdc[23635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977898, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:18.225893 23652 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:58:18.226891 23652 webserver.cc:460] Webserver started at http://127.0.0.1:39991/ using document root <none> and password file <none>
I20260806 00:58:18.227488 23652 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:18.227609 23652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:18.227862 23652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:18.230017 23652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "1d80f51f7f5343b19f5516a6351faf12"
format_stamp: "Formatted at 2026-08-06 00:58:18 on dist-test-slave-sb2z"
server_key: "c8edc6616dba033354538d7f16575719"
server_key_iv: "775aba72ab83026374be00954e9d2b72"
server_key_version: "encryptionkey@0"
I20260806 00:58:18.230624 23652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "1d80f51f7f5343b19f5516a6351faf12"
format_stamp: "Formatted at 2026-08-06 00:58:18 on dist-test-slave-sb2z"
server_key: "c8edc6616dba033354538d7f16575719"
server_key_iv: "775aba72ab83026374be00954e9d2b72"
server_key_version: "encryptionkey@0"
I20260806 00:58:18.234604 23652 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 00:58:18.237243 23676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:18.238559 23652 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:58:18.238723 23652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "1d80f51f7f5343b19f5516a6351faf12"
format_stamp: "Formatted at 2026-08-06 00:58:18 on dist-test-slave-sb2z"
server_key: "c8edc6616dba033354538d7f16575719"
server_key_iv: "775aba72ab83026374be00954e9d2b72"
server_key_version: "encryptionkey@0"
I20260806 00:58:18.238886 23652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:18.244012 23652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:18.247162 23652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:18.247318 23652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:18.254982 23652 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43489
I20260806 00:58:18.255013 23751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43489 every 8 connection(s)
I20260806 00:58:18.256043 23652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:18.259032 23752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:18.260259 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 23652
I20260806 00:58:18.260419 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:18.260713 27959 external_mini_cluster.cc:1428] Setting key e2c7ec4b479029197e79a7553c7d7d33
I20260806 00:58:18.266026 23752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12: Bootstrap starting.
I20260806 00:58:18.266300 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:18.262209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52993 (local address 127.0.0.1:43489)
0806 00:58:18.262622 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:18.262638 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:18.262673 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:58:18.263296 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:18.263312 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:18.263660 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:18.263975 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:18.264912 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:18.264917 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:18.264929 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:18.264940 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:18.265018 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:18.265055 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:18.265187 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:18.265327 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 00:58:18.265495 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":137,"threads_started":1}
I20260806 00:58:18.268617 23752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:18.269183 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:18.269527 23752 log.cc:824] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:18.271891 23752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12: No bootstrap required, opened a new log
I20260806 00:58:18.273239 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:18.271334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40773 (local address 127.0.0.1:43489)
0806 00:58:18.271470 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:18.271474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:18.271742 (+ 268us) server_negotiation.cc:407] Connection header received
0806 00:58:18.271842 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:18.271844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:18.271875 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:18.271951 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:18.272694 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:18.272698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:18.272700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:18.272710 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:18.272734 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:18.272744 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:18.272815 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:18.272977 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 00:58:18.273041 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:58:18.276814 23752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d80f51f7f5343b19f5516a6351faf12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43489 } }
I20260806 00:58:18.277063 23752 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:18.277149 23752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d80f51f7f5343b19f5516a6351faf12, State: Initialized, Role: FOLLOWER
I20260806 00:58:18.277609 23752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d80f51f7f5343b19f5516a6351faf12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43489 } }
I20260806 00:58:18.277774 23752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:18.277848 23752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:18.277979 23752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:18.279168 23752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d80f51f7f5343b19f5516a6351faf12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43489 } }
I20260806 00:58:18.279610 23752 leader_election.cc:302] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d80f51f7f5343b19f5516a6351faf12; no voters:
I20260806 00:58:18.279976 23752 leader_election.cc:288] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:18.280141 23769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:18.280424 23769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [term 1 LEADER]: Becoming Leader. State: Replica: 1d80f51f7f5343b19f5516a6351faf12, State: Running, Role: LEADER
I20260806 00:58:18.280786 23769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d80f51f7f5343b19f5516a6351faf12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43489 } }
I20260806 00:58:18.281407 23752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:18.282665 23774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d80f51f7f5343b19f5516a6351faf12. Latest consensus state: current_term: 1 leader_uuid: "1d80f51f7f5343b19f5516a6351faf12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d80f51f7f5343b19f5516a6351faf12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43489 } } }
I20260806 00:58:18.282822 23774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:18.283255 23783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:18.283211 23773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d80f51f7f5343b19f5516a6351faf12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d80f51f7f5343b19f5516a6351faf12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43489 } } }
I20260806 00:58:18.284075 23773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:18.286593 23783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:18.293057 23783 catalog_manager.cc:1347] Generated new cluster ID: 7035f3c7d3934863a92e693d2609cd3b
I20260806 00:58:18.293143 23783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:18.318415 23783 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:18.318859 23783 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:18.326835 23783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d80f51f7f5343b19f5516a6351faf12: Generated new TSK 0
I20260806 00:58:18.327771 23783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:18.341287 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 23652
2026-08-06T00:58:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1606 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:18 dist-test-slave-sb2z krb5kdc[23798](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:18 dist-test-slave-sb2z krb5kdc[23798](info): set up 2 sockets
Aug 06 00:58:18 dist-test-slave-sb2z krb5kdc[23798](info): commencing operation
krb5kdc: starting...
W20260806 00:58:19.620079 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:19 dist-test-slave-sb2z krb5kdc[23798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:58:19.759835 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33319
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:19.883363 23818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:19.883741 23818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:19.883792 23818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:19.887943 23818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:19.888015 23818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:19.888036 23818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:19.888054 23818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:19.888070 23818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:19.893537 23818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:19.894636 23818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:19.895697 23818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:19.904700 23825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:19.904695 23824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:19.904695 23828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:19.904855 23818 server_base.cc:1034] running on GCE node
I20260806 00:58:19.905602 23818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:19.906723 23818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:19.907927 23818 hybrid_clock.cc:648] HybridClock initialized: now 1785977899907906 us; error 31 us; skew 500 ppm
Aug 06 00:58:19 dist-test-slave-sb2z krb5kdc[23798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977899, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:19.910984 23818 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:58:19.912031 23818 webserver.cc:460] Webserver started at http://127.0.0.1:37709/ using document root <none> and password file <none>
I20260806 00:58:19.912652 23818 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:19.912750 23818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:19.913003 23818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:19.915118 23818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "58b31eead20d4c8e9ad24102b4e078ce"
format_stamp: "Formatted at 2026-08-06 00:58:19 on dist-test-slave-sb2z"
server_key: "20082aba2770640a594289893d3f586f"
server_key_iv: "d7a688fe7ebbf0ca3c9e330071502b07"
server_key_version: "encryptionkey@0"
I20260806 00:58:19.915781 23818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "58b31eead20d4c8e9ad24102b4e078ce"
format_stamp: "Formatted at 2026-08-06 00:58:19 on dist-test-slave-sb2z"
server_key: "20082aba2770640a594289893d3f586f"
server_key_iv: "d7a688fe7ebbf0ca3c9e330071502b07"
server_key_version: "encryptionkey@0"
I20260806 00:58:19.919781 23818 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:58:19.922318 23837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:19.923840 23818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:19.923995 23818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "58b31eead20d4c8e9ad24102b4e078ce"
format_stamp: "Formatted at 2026-08-06 00:58:19 on dist-test-slave-sb2z"
server_key: "20082aba2770640a594289893d3f586f"
server_key_iv: "d7a688fe7ebbf0ca3c9e330071502b07"
server_key_version: "encryptionkey@0"
I20260806 00:58:19.924118 23818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:19.935223 23818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:19.938653 23818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:19.938871 23818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:19.946923 23818 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33319
I20260806 00:58:19.946935 23912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33319 every 8 connection(s)
I20260806 00:58:19.948089 23818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:19.950999 23913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:19.956884 23913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce: Bootstrap starting.
I20260806 00:58:19.958338 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 23818
I20260806 00:58:19.958511 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:19.958811 27959 external_mini_cluster.cc:1428] Setting key 0a2200900d5a4e207368a3a317157245
I20260806 00:58:19.959803 23913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:19.960670 23913 log.cc:824] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:19.962878 23913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce: No bootstrap required, opened a new log
I20260806 00:58:19.967252 23913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58b31eead20d4c8e9ad24102b4e078ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33319 } }
I20260806 00:58:19.967592 23913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:19.967660 23913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58b31eead20d4c8e9ad24102b4e078ce, State: Initialized, Role: FOLLOWER
I20260806 00:58:19.968207 23913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58b31eead20d4c8e9ad24102b4e078ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33319 } }
I20260806 00:58:19.968345 23913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:19.968418 23913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:19.968518 23913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [term 0 FOLLOWER]: Advancing to term 1
Aug 06 00:58:19 dist-test-slave-sb2z krb5kdc[23798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 00:58:19.969684 23913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58b31eead20d4c8e9ad24102b4e078ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33319 } }
I20260806 00:58:19.970119 23913 leader_election.cc:302] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58b31eead20d4c8e9ad24102b4e078ce; no voters:
I20260806 00:58:19.970551 23913 leader_election.cc:288] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:19.970659 23919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:19.970897 23919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [term 1 LEADER]: Becoming Leader. State: Replica: 58b31eead20d4c8e9ad24102b4e078ce, State: Running, Role: LEADER
I20260806 00:58:19.971251 23919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58b31eead20d4c8e9ad24102b4e078ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33319 } }
I20260806 00:58:19.972056 23913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:19.972530 23923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58b31eead20d4c8e9ad24102b4e078ce. Latest consensus state: current_term: 1 leader_uuid: "58b31eead20d4c8e9ad24102b4e078ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58b31eead20d4c8e9ad24102b4e078ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33319 } } }
I20260806 00:58:19.972719 23923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:19.972985 23922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58b31eead20d4c8e9ad24102b4e078ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58b31eead20d4c8e9ad24102b4e078ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33319 } } }
I20260806 00:58:19.973129 23922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:19.974156 23928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:19.975689 23917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:19.960243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:42339 (local address 127.0.0.1:33319)
0806 00:58:19.960737 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:19.960753 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:19.960810 (+ 57us) server_negotiation.cc:407] Connection header received
0806 00:58:19.961715 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:19.961735 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:19.962065 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:19.962348 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:19.963332 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:19.964409 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:19.965664 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:19.966152 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:19.966270 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:19.969475 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:19.969484 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:19.969505 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:19.969543 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:19.971720 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:19.972349 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:19.972355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:19.972361 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:19.972510 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:19.972997 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:19.973002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:19.973007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:19.973584 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:19.973847 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:19.974471 (+ 624us) server_negotiation.cc:299] Negotiation successful
0806 00:58:19.974712 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":169,"threads_started":1}
I20260806 00:58:19.977550 23928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:19.978660 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:19.983915 23928 catalog_manager.cc:1347] Generated new cluster ID: ce73aafdf8e547f3a2528aa3920e09ab
I20260806 00:58:19.983985 23928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:19.990674 23917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:19.980836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:37775 (local address 127.0.0.1:33319)
0806 00:58:19.981907 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:19.981912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:19.981927 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:58:19.981965 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:19.981967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:19.982018 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:19.982128 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:19.983133 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:19.983924 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:19.984786 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:19.985190 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:19.985290 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:19.987046 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:19.987049 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:19.987052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:19.987083 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:19.988803 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:19.989365 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:19.989369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:19.989371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:19.989449 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:19.989793 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:19.989796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:19.989797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:19.989981 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:19.990130 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:19.990408 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 00:58:19.990501 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1004}
I20260806 00:58:20.005704 23928 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:20.006640 23928 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:20.013571 23928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58b31eead20d4c8e9ad24102b4e078ce: Generated new TSK 0
I20260806 00:58:20.014349 23928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:20.027228 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 23818
W20260806 00:58:20.043912 23796 connection.cc:535] client connection to 127.0.0.1:33319 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33319): Connection reset by peer (error 104)
2026-08-06T00:58:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1687 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:20 dist-test-slave-sb2z krb5kdc[23951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:20 dist-test-slave-sb2z krb5kdc[23951](info): set up 2 sockets
Aug 06 00:58:20 dist-test-slave-sb2z krb5kdc[23951](info): commencing operation
krb5kdc: starting...
W20260806 00:58:21.250634 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:21 dist-test-slave-sb2z krb5kdc[23951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:58:21.389284 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36387
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:21.504796 23970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:21.505143 23970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:21.505232 23970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:21.509358 23970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:21.509471 23970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:21.509513 23970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:21.509577 23970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:21.509619 23970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:21.515134 23970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:21.516350 23970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:21.517447 23970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:21.526636 23977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:21.526652 23981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:21.526652 23979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:21.526762 23970 server_base.cc:1034] running on GCE node
I20260806 00:58:21.527585 23970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:21.528811 23970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:21.529994 23970 hybrid_clock.cc:648] HybridClock initialized: now 1785977901529962 us; error 49 us; skew 500 ppm
Aug 06 00:58:21 dist-test-slave-sb2z krb5kdc[23951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977901, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:21.533053 23970 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:58:21.534026 23970 webserver.cc:460] Webserver started at http://127.0.0.1:38341/ using document root <none> and password file <none>
I20260806 00:58:21.534591 23970 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:21.534654 23970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:21.534828 23970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:21.537102 23970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "988680ccf4d14622adebcaf330fac745"
format_stamp: "Formatted at 2026-08-06 00:58:21 on dist-test-slave-sb2z"
server_key: "b770d789e996f5dd021bda9ad0f91c22"
server_key_iv: "2b207eca43eb06d45795d2ae3fc7a8c4"
server_key_version: "encryptionkey@0"
I20260806 00:58:21.537659 23970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "988680ccf4d14622adebcaf330fac745"
format_stamp: "Formatted at 2026-08-06 00:58:21 on dist-test-slave-sb2z"
server_key: "b770d789e996f5dd021bda9ad0f91c22"
server_key_iv: "2b207eca43eb06d45795d2ae3fc7a8c4"
server_key_version: "encryptionkey@0"
I20260806 00:58:21.541715 23970 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:58:21.544584 23990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:21.546051 23970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:21.546237 23970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "988680ccf4d14622adebcaf330fac745"
format_stamp: "Formatted at 2026-08-06 00:58:21 on dist-test-slave-sb2z"
server_key: "b770d789e996f5dd021bda9ad0f91c22"
server_key_iv: "2b207eca43eb06d45795d2ae3fc7a8c4"
server_key_version: "encryptionkey@0"
I20260806 00:58:21.546381 23970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:21.563519 23970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:21.566908 23970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:21.567165 23970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:21.574819 23970 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36387
I20260806 00:58:21.574867 24062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36387 every 8 connection(s)
I20260806 00:58:21.575875 23970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:21.577466 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 23970
I20260806 00:58:21.577642 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:21.577947 27959 external_mini_cluster.cc:1428] Setting key 9d5afda3c3bcdff72831f0b0fad33608
I20260806 00:58:21.579274 24063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:21.586373 24063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745: Bootstrap starting.
I20260806 00:58:21.588406 24067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:21.579415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:52155 (local address 127.0.0.1:36387)
0806 00:58:21.579935 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:21.579954 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:21.579996 (+ 42us) server_negotiation.cc:407] Connection header received
0806 00:58:21.580856 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:21.580881 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:21.581339 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:21.581704 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:21.582123 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:21.583649 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:21.584522 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:21.585234 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:21.585407 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:21.586330 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:21.586336 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:21.586352 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:21.586847 (+ 495us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:21.586914 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:21.586946 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:21.587082 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:21.587266 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 00:58:21.587395 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":139,"threads_started":1}
I20260806 00:58:21.589156 24063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:21.589919 24063 log.cc:824] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:21.591418 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:21.592114 24063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745: No bootstrap required, opened a new log
I20260806 00:58:21.596465 24063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988680ccf4d14622adebcaf330fac745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36387 } }
I20260806 00:58:21.596769 24063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:21.596856 24063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 988680ccf4d14622adebcaf330fac745, State: Initialized, Role: FOLLOWER
I20260806 00:58:21.597366 24063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988680ccf4d14622adebcaf330fac745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36387 } }
I20260806 00:58:21.597551 24063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:21.597647 24063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:21.597752 24063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:21.598099 24067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:21.593638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:33405 (local address 127.0.0.1:36387)
0806 00:58:21.593751 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:21.593755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:21.593911 (+ 156us) server_negotiation.cc:407] Connection header received
0806 00:58:21.594004 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:21.594007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:21.594064 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:21.594172 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:21.594406 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:21.595187 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:21.596036 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:21.596357 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:21.596466 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:21.597338 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:21.597341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:21.597343 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:21.597354 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:21.597378 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:21.597388 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:21.597452 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:21.597970 (+ 518us) server_negotiation.cc:299] Negotiation successful
0806 00:58:21.598005 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:58:21.598789 24063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988680ccf4d14622adebcaf330fac745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36387 } }
I20260806 00:58:21.599159 24063 leader_election.cc:302] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 988680ccf4d14622adebcaf330fac745; no voters:
I20260806 00:58:21.599493 24063 leader_election.cc:288] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:21.599660 24079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:21.599903 24079 raft_consensus.cc:686] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [term 1 LEADER]: Becoming Leader. State: Replica: 988680ccf4d14622adebcaf330fac745, State: Running, Role: LEADER
I20260806 00:58:21.600242 24079 consensus_queue.cc:237] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988680ccf4d14622adebcaf330fac745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36387 } }
I20260806 00:58:21.600739 24063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:21.602015 24081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 988680ccf4d14622adebcaf330fac745. Latest consensus state: current_term: 1 leader_uuid: "988680ccf4d14622adebcaf330fac745" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988680ccf4d14622adebcaf330fac745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36387 } } }
I20260806 00:58:21.602170 24081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:21.602006 24080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "988680ccf4d14622adebcaf330fac745" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988680ccf4d14622adebcaf330fac745" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36387 } } }
I20260806 00:58:21.602520 24080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:21.602586 24092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:21.605872 24092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:21.611915 24092 catalog_manager.cc:1347] Generated new cluster ID: dabaf63e0fa341e7a19bc0bd1f702c47
I20260806 00:58:21.612004 24092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:21.629449 24092 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:21.630683 24092 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:21.638792 24092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 988680ccf4d14622adebcaf330fac745: Generated new TSK 0
I20260806 00:58:21.639667 24092 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:21.648192 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 23970
W20260806 00:58:21.658829 23949 connection.cc:535] client connection to 127.0.0.1:36387 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36387): Connection reset by peer (error 104)
2026-08-06T00:58:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1614 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:21 dist-test-slave-sb2z krb5kdc[24108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:21 dist-test-slave-sb2z krb5kdc[24108](info): set up 2 sockets
Aug 06 00:58:21 dist-test-slave-sb2z krb5kdc[24108](info): commencing operation
krb5kdc: starting...
W20260806 00:58:22.842967 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:22 dist-test-slave-sb2z krb5kdc[24108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:58:22.983326 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42509
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:23.102218 24124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:23.102540 24124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:23.102619 24124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:23.107393 24124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:23.107489 24124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:23.107527 24124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:23.107586 24124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:23.107625 24124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:23.113643 24124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:23.115150 24124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:23.116263 24124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:58:23.125458 24124 server_base.cc:1034] running on GCE node
W20260806 00:58:23.125417 24136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:23.125435 24129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:23.125624 24131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:23.126212 24124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:23.127396 24124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:23.128580 24124 hybrid_clock.cc:648] HybridClock initialized: now 1785977903128548 us; error 43 us; skew 500 ppm
Aug 06 00:58:23 dist-test-slave-sb2z krb5kdc[24108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977903, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:23.131420 24124 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:58:23.132364 24124 webserver.cc:460] Webserver started at http://127.0.0.1:46163/ using document root <none> and password file <none>
I20260806 00:58:23.132980 24124 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:23.133052 24124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:23.133279 24124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:23.135377 24124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "b646344fc57948868f05747ed35e6b11"
format_stamp: "Formatted at 2026-08-06 00:58:23 on dist-test-slave-sb2z"
server_key: "6468bc64c6c1d286f7154408c1d374ec"
server_key_iv: "0a81a57b4cfe83779f257a23628a0ff7"
server_key_version: "encryptionkey@0"
I20260806 00:58:23.136091 24124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "b646344fc57948868f05747ed35e6b11"
format_stamp: "Formatted at 2026-08-06 00:58:23 on dist-test-slave-sb2z"
server_key: "6468bc64c6c1d286f7154408c1d374ec"
server_key_iv: "0a81a57b4cfe83779f257a23628a0ff7"
server_key_version: "encryptionkey@0"
I20260806 00:58:23.139941 24124 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:58:23.142505 24144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:23.144053 24124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:23.144222 24124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "b646344fc57948868f05747ed35e6b11"
format_stamp: "Formatted at 2026-08-06 00:58:23 on dist-test-slave-sb2z"
server_key: "6468bc64c6c1d286f7154408c1d374ec"
server_key_iv: "0a81a57b4cfe83779f257a23628a0ff7"
server_key_version: "encryptionkey@0"
I20260806 00:58:23.144346 24124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:23.149456 24124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:23.152988 24124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:23.153316 24124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:23.161696 24124 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42509
I20260806 00:58:23.161733 24221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42509 every 8 connection(s)
I20260806 00:58:23.162756 24124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:23.165773 24222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:23.171351 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 24124
I20260806 00:58:23.171578 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:23.171922 27959 external_mini_cluster.cc:1428] Setting key 4e42964eecebf8acdd3f6e22ebf95ec6
I20260806 00:58:23.171950 24222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11: Bootstrap starting.
I20260806 00:58:23.174598 24222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:23.175369 24222 log.cc:824] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:23.177538 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:23.173445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:58269 (local address 127.0.0.1:42509)
0806 00:58:23.173918 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:23.173933 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:23.173969 (+ 36us) server_negotiation.cc:407] Connection header received
0806 00:58:23.174761 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:23.174780 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:23.175072 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:23.175417 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:23.176316 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:23.176328 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:23.176339 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:23.176350 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:23.176404 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:23.176423 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:23.176560 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:23.176671 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 00:58:23.176842 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":132,"threads_started":1}
I20260806 00:58:23.177568 24222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11: No bootstrap required, opened a new log
I20260806 00:58:23.180641 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:23.181514 24222 raft_consensus.cc:348] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b646344fc57948868f05747ed35e6b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260806 00:58:23.181772 24222 raft_consensus.cc:374] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:23.181852 24222 raft_consensus.cc:729] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b646344fc57948868f05747ed35e6b11, State: Initialized, Role: FOLLOWER
I20260806 00:58:23.182389 24222 consensus_queue.cc:260] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b646344fc57948868f05747ed35e6b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260806 00:58:23.182550 24222 raft_consensus.cc:388] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:23.182643 24222 raft_consensus.cc:482] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:23.182754 24222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:23.184062 24222 raft_consensus.cc:504] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b646344fc57948868f05747ed35e6b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260806 00:58:23.184458 24222 leader_election.cc:302] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b646344fc57948868f05747ed35e6b11; no voters:
I20260806 00:58:23.184861 24222 leader_election.cc:288] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:23.184973 24236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:23.185017 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:23.183372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:55809 (local address 127.0.0.1:42509)
0806 00:58:23.183512 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:23.183516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:23.183548 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:58:23.183585 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:23.183588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:23.183622 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:23.183711 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:23.184566 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:23.184570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:23.184575 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:23.184589 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:23.184620 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:23.184632 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:23.184724 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:23.184812 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 00:58:23.184856 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:58:23.185308 24236 raft_consensus.cc:686] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [term 1 LEADER]: Becoming Leader. State: Replica: b646344fc57948868f05747ed35e6b11, State: Running, Role: LEADER
I20260806 00:58:23.185631 24236 consensus_queue.cc:237] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b646344fc57948868f05747ed35e6b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260806 00:58:23.186337 24222 sys_catalog.cc:564] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:23.187711 24238 sys_catalog.cc:455] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b646344fc57948868f05747ed35e6b11. Latest consensus state: current_term: 1 leader_uuid: "b646344fc57948868f05747ed35e6b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b646344fc57948868f05747ed35e6b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } } }
I20260806 00:58:23.187844 24238 sys_catalog.cc:458] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:23.187721 24237 sys_catalog.cc:455] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b646344fc57948868f05747ed35e6b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b646344fc57948868f05747ed35e6b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } } }
I20260806 00:58:23.188021 24237 sys_catalog.cc:458] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11 [sys.catalog]: This master's current role is: LEADER
W20260806 00:58:23.191222 24255 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:58:23.191360 24255 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:58:23.191457 24249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:23.192198 24249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:23.198326 24249 catalog_manager.cc:1347] Generated new cluster ID: 344f932b1c624c609e04374b35f162fe
I20260806 00:58:23.198412 24249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:23.228964 24249 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:23.229377 24249 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:23.234325 24249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b646344fc57948868f05747ed35e6b11: Generated new TSK 0
I20260806 00:58:23.235260 24249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:23.238238 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 24124
2026-08-06T00:58:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1594 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:23 dist-test-slave-sb2z krb5kdc[24265](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:23 dist-test-slave-sb2z krb5kdc[24265](info): set up 2 sockets
Aug 06 00:58:23 dist-test-slave-sb2z krb5kdc[24265](info): commencing operation
krb5kdc: starting...
W20260806 00:58:24.491070 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:24 dist-test-slave-sb2z krb5kdc[24265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:58:24.640651 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33309
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:24.760466 24285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:24.760774 24285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:24.760821 24285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:24.764969 24285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:24.765044 24285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:24.765064 24285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:24.765082 24285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:24.765098 24285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:24.770566 24285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33309
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:24.771715 24285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:24.772740 24285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:24.782289 24291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:24.782284 24290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:24.782516 24285 server_base.cc:1034] running on GCE node
W20260806 00:58:24.782289 24293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:24.783146 24285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:24.784385 24285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:24.785593 24285 hybrid_clock.cc:648] HybridClock initialized: now 1785977904785580 us; error 32 us; skew 500 ppm
Aug 06 00:58:24 dist-test-slave-sb2z krb5kdc[24265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977904, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:24.794500 24285 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:58:24.795451 24285 webserver.cc:460] Webserver started at http://127.0.0.1:34231/ using document root <none> and password file <none>
I20260806 00:58:24.796087 24285 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:24.796155 24285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:24.796327 24285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:24.798430 24285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "62e74aea9dd2499ba277f2179f23a7e0"
format_stamp: "Formatted at 2026-08-06 00:58:24 on dist-test-slave-sb2z"
server_key: "3e2782acf10d73445df21814e18d0887"
server_key_iv: "a1ef13a8eb04f5ede538906e72eb7ef5"
server_key_version: "encryptionkey@0"
I20260806 00:58:24.798954 24285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "62e74aea9dd2499ba277f2179f23a7e0"
format_stamp: "Formatted at 2026-08-06 00:58:24 on dist-test-slave-sb2z"
server_key: "3e2782acf10d73445df21814e18d0887"
server_key_iv: "a1ef13a8eb04f5ede538906e72eb7ef5"
server_key_version: "encryptionkey@0"
I20260806 00:58:24.802911 24285 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:58:24.805408 24307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:24.806764 24285 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:58:24.806879 24285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "62e74aea9dd2499ba277f2179f23a7e0"
format_stamp: "Formatted at 2026-08-06 00:58:24 on dist-test-slave-sb2z"
server_key: "3e2782acf10d73445df21814e18d0887"
server_key_iv: "a1ef13a8eb04f5ede538906e72eb7ef5"
server_key_version: "encryptionkey@0"
I20260806 00:58:24.806972 24285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:24.824512 24285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:24.827788 24285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:24.827958 24285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:24.836014 24285 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33309
I20260806 00:58:24.836145 24375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33309 every 8 connection(s)
I20260806 00:58:24.837080 24285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:24.838991 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 24285
I20260806 00:58:24.839263 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:24.839668 27959 external_mini_cluster.cc:1428] Setting key 140da886db27596e77d8323ecba722ad
I20260806 00:58:24.840304 24376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:24.847461 24376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0: Bootstrap starting.
Aug 06 00:58:24 dist-test-slave-sb2z krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 00:58:24.850235 24376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:24.851055 24376 log.cc:824] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:24.853461 24376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0: No bootstrap required, opened a new log
I20260806 00:58:24.856422 24384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:24.841104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:36953 (local address 127.0.0.1:33309)
0806 00:58:24.841572 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:24.841589 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:24.841627 (+ 38us) server_negotiation.cc:407] Connection header received
0806 00:58:24.842293 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:24.842309 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:24.842662 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:24.842944 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:24.843994 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:24.845013 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:24.846258 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:24.846781 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:24.846912 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:24.850128 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:24.850133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:24.850148 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:24.850182 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:24.852329 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:24.853111 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:24.853117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:24.853124 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:24.853297 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:24.853829 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:24.853847 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:24.853849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:24.854427 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:24.854688 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:24.855000 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 00:58:24.855256 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":142,"threads_started":1}
I20260806 00:58:24.858583 24376 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e74aea9dd2499ba277f2179f23a7e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33309 } }
I20260806 00:58:24.858891 24376 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:24.858979 24376 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e74aea9dd2499ba277f2179f23a7e0, State: Initialized, Role: FOLLOWER
I20260806 00:58:24.859489 24376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e74aea9dd2499ba277f2179f23a7e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33309 } }
I20260806 00:58:24.859700 24376 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:24.859794 24376 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:24.859900 24376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:24.860294 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:24.861128 24376 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e74aea9dd2499ba277f2179f23a7e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33309 } }
I20260806 00:58:24.862144 24376 leader_election.cc:302] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62e74aea9dd2499ba277f2179f23a7e0; no voters:
I20260806 00:58:24.862582 24376 leader_election.cc:288] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:24.862744 24392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:24.863086 24392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [term 1 LEADER]: Becoming Leader. State: Replica: 62e74aea9dd2499ba277f2179f23a7e0, State: Running, Role: LEADER
I20260806 00:58:24.863547 24392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e74aea9dd2499ba277f2179f23a7e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33309 } }
I20260806 00:58:24.864224 24376 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:24.865475 24394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62e74aea9dd2499ba277f2179f23a7e0. Latest consensus state: current_term: 1 leader_uuid: "62e74aea9dd2499ba277f2179f23a7e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e74aea9dd2499ba277f2179f23a7e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33309 } } }
I20260806 00:58:24.865615 24394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:24.866113 24396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62e74aea9dd2499ba277f2179f23a7e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e74aea9dd2499ba277f2179f23a7e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33309 } } }
I20260806 00:58:24.866284 24396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0 [sys.catalog]: This master's current role is: LEADER
W20260806 00:58:24.869738 24416 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:58:24.869827 24416 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:58:24.869979 24406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:24.870893 24406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:24.873507 24384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:24.862710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:46951 (local address 127.0.0.1:33309)
0806 00:58:24.862840 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:24.862846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:24.863219 (+ 373us) server_negotiation.cc:407] Connection header received
0806 00:58:24.863677 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:24.863680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:24.863738 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:24.863829 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:24.864715 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:24.865562 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:24.866860 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:24.867224 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:24.867366 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:24.869371 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:24.869375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:24.869378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:24.869406 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:24.871217 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:24.871852 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:24.871856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:24.871858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:24.871955 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:24.872434 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:24.872438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:24.872440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:24.872687 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:24.872846 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:24.873225 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 00:58:24.873339 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":34}
I20260806 00:58:24.877544 24406 catalog_manager.cc:1347] Generated new cluster ID: 8b5bd7bfafd547a1869c14a0eebe2bf0
I20260806 00:58:24.877627 24406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:24.907944 24406 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:24.909055 24406 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:24.931167 24406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62e74aea9dd2499ba277f2179f23a7e0: Generated new TSK 0
I20260806 00:58:24.932085 24406 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:24.948876 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 24285
W20260806 00:58:24.959741 24262 connection.cc:535] client connection to 127.0.0.1:33309 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33309): Connection reset by peer (error 104)
2026-08-06T00:58:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1706 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:25 dist-test-slave-sb2z krb5kdc[24424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:25 dist-test-slave-sb2z krb5kdc[24424](info): set up 2 sockets
Aug 06 00:58:25 dist-test-slave-sb2z krb5kdc[24424](info): commencing operation
krb5kdc: starting...
W20260806 00:58:26.161217 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:26 dist-test-slave-sb2z krb5kdc[24424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:58:26.303742 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34713
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:26.421663 24438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:26.421942 24438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:26.421984 24438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:26.426044 24438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:26.426115 24438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:26.426134 24438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:26.426152 24438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:26.426167 24438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:26.431699 24438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34713
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:26.432718 24438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:26.433692 24438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:26.442871 24447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:26.442885 24444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:26.442883 24445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:26.443140 24438 server_base.cc:1034] running on GCE node
I20260806 00:58:26.443892 24438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:26.445000 24438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:26.446260 24438 hybrid_clock.cc:648] HybridClock initialized: now 1785977906446243 us; error 33 us; skew 500 ppm
Aug 06 00:58:26 dist-test-slave-sb2z krb5kdc[24424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977906, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:26.449594 24438 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:58:26.450568 24438 webserver.cc:460] Webserver started at http://127.0.0.1:46245/ using document root <none> and password file <none>
I20260806 00:58:26.451200 24438 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:26.451299 24438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:26.451596 24438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:26.453765 24438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "1cb66b1143534969b448764940adf9a0"
format_stamp: "Formatted at 2026-08-06 00:58:26 on dist-test-slave-sb2z"
server_key: "91f46db385de8206e7eff95e6c225e2f"
server_key_iv: "a825acdae0ade345d4233258d92fb0aa"
server_key_version: "encryptionkey@0"
I20260806 00:58:26.454371 24438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "1cb66b1143534969b448764940adf9a0"
format_stamp: "Formatted at 2026-08-06 00:58:26 on dist-test-slave-sb2z"
server_key: "91f46db385de8206e7eff95e6c225e2f"
server_key_iv: "a825acdae0ade345d4233258d92fb0aa"
server_key_version: "encryptionkey@0"
I20260806 00:58:26.458400 24438 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 00:58:26.461040 24458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:26.462450 24438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 00:58:26.462648 24438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "1cb66b1143534969b448764940adf9a0"
format_stamp: "Formatted at 2026-08-06 00:58:26 on dist-test-slave-sb2z"
server_key: "91f46db385de8206e7eff95e6c225e2f"
server_key_iv: "a825acdae0ade345d4233258d92fb0aa"
server_key_version: "encryptionkey@0"
I20260806 00:58:26.462788 24438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:26.481325 24438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:26.485185 24438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:26.485433 24438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:26.493618 24438 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34713
I20260806 00:58:26.493682 24532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34713 every 8 connection(s)
I20260806 00:58:26.494771 24438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:26.497825 24533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:26.501888 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 24438
I20260806 00:58:26.502043 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:26.502379 27959 external_mini_cluster.cc:1428] Setting key bbde4799aff4a82ccdc5d37446087405
I20260806 00:58:26.504133 24533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0: Bootstrap starting.
I20260806 00:58:26.506636 24533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:26.507519 24533 log.cc:824] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:26.509902 24533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0: No bootstrap required, opened a new log
I20260806 00:58:26.512399 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:26.503889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:33907 (local address 127.0.0.1:34713)
0806 00:58:26.504324 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:26.504347 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:26.504384 (+ 37us) server_negotiation.cc:407] Connection header received
0806 00:58:26.505145 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:26.505163 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:26.505516 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:26.505796 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:26.506127 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:26.507173 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:26.508376 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:26.508953 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:26.509100 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:26.510155 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:26.510160 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:26.510174 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:26.510579 (+ 405us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:26.510637 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:26.510661 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:26.510776 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:26.511079 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 00:58:26.511244 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":146,"threads_started":1}
I20260806 00:58:26.513355 24533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb66b1143534969b448764940adf9a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34713 } }
I20260806 00:58:26.513665 24533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:26.513733 24533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb66b1143534969b448764940adf9a0, State: Initialized, Role: FOLLOWER
I20260806 00:58:26.514218 24533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb66b1143534969b448764940adf9a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34713 } }
I20260806 00:58:26.514391 24533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:26.514459 24533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:26.514591 24533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:26.515887 24533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb66b1143534969b448764940adf9a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34713 } }
I20260806 00:58:26.516254 24533 leader_election.cc:302] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cb66b1143534969b448764940adf9a0; no voters:
I20260806 00:58:26.516403 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:26.516662 24533 leader_election.cc:288] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:26.516778 24539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:26.517051 24539 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [term 1 LEADER]: Becoming Leader. State: Replica: 1cb66b1143534969b448764940adf9a0, State: Running, Role: LEADER
I20260806 00:58:26.517447 24539 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb66b1143534969b448764940adf9a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34713 } }
I20260806 00:58:26.518181 24533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:26.519243 24542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cb66b1143534969b448764940adf9a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb66b1143534969b448764940adf9a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34713 } } }
I20260806 00:58:26.519371 24542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:26.519649 24542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cb66b1143534969b448764940adf9a0. Latest consensus state: current_term: 1 leader_uuid: "1cb66b1143534969b448764940adf9a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb66b1143534969b448764940adf9a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34713 } } }
I20260806 00:58:26.519730 24542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:26.520066 24557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:26.524158 24557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:26.526423 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:26.521070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:45465 (local address 127.0.0.1:34713)
0806 00:58:26.521196 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:26.521201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:26.521401 (+ 200us) server_negotiation.cc:407] Connection header received
0806 00:58:26.521587 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:26.521591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:26.521649 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:26.521759 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:26.522062 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:26.522838 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:26.524066 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:26.524400 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:26.524510 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:26.525719 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:26.525724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:26.525727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:26.525742 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:26.525772 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:26.525786 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:26.525873 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:26.526236 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 00:58:26.526277 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:58:26.530357 24557 catalog_manager.cc:1347] Generated new cluster ID: 5d1e67181f1e4f458e5af3601e04bacc
I20260806 00:58:26.530441 24557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:26.538125 24557 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:26.539351 24557 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:26.544919 24557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cb66b1143534969b448764940adf9a0: Generated new TSK 0
I20260806 00:58:26.545763 24557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:26.555866 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 24438
W20260806 00:58:26.566717 24422 connection.cc:535] client connection to 127.0.0.1:34713 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34713): Connection reset by peer (error 104)
2026-08-06T00:58:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1607 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:26 dist-test-slave-sb2z krb5kdc[24579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:26 dist-test-slave-sb2z krb5kdc[24579](info): set up 2 sockets
Aug 06 00:58:26 dist-test-slave-sb2z krb5kdc[24579](info): commencing operation
krb5kdc: starting...
W20260806 00:58:27.826117 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:27 dist-test-slave-sb2z krb5kdc[24579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:58:27.971614 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42793
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:28.091542 24599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:28.091840 24599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:28.091885 24599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:28.096091 24599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:28.096165 24599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:28.096185 24599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:28.096203 24599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:28.096269 24599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:28.101881 24599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42793
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:28.103127 24599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:28.104256 24599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:28.113785 24607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:28.113858 24608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:28.113986 24599 server_base.cc:1034] running on GCE node
W20260806 00:58:28.113726 24610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:28.114812 24599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:28.116187 24599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:28.117411 24599 hybrid_clock.cc:648] HybridClock initialized: now 1785977908117379 us; error 50 us; skew 500 ppm
Aug 06 00:58:28 dist-test-slave-sb2z krb5kdc[24579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977908, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:28.121160 24599 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:58:28.122200 24599 webserver.cc:460] Webserver started at http://127.0.0.1:46089/ using document root <none> and password file <none>
I20260806 00:58:28.122831 24599 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:28.122936 24599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:28.123199 24599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:28.125655 24599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "ebe93d5761f449238285796369322a6a"
format_stamp: "Formatted at 2026-08-06 00:58:28 on dist-test-slave-sb2z"
server_key: "efd1cb67fd2d74f18aabf42907d8cd95"
server_key_iv: "071fe3debba60cb0a605f85f3c764afd"
server_key_version: "encryptionkey@0"
I20260806 00:58:28.126322 24599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "ebe93d5761f449238285796369322a6a"
format_stamp: "Formatted at 2026-08-06 00:58:28 on dist-test-slave-sb2z"
server_key: "efd1cb67fd2d74f18aabf42907d8cd95"
server_key_iv: "071fe3debba60cb0a605f85f3c764afd"
server_key_version: "encryptionkey@0"
I20260806 00:58:28.130765 24599 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:58:28.133667 24618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:28.135128 24599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:58:28.135397 24599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "ebe93d5761f449238285796369322a6a"
format_stamp: "Formatted at 2026-08-06 00:58:28 on dist-test-slave-sb2z"
server_key: "efd1cb67fd2d74f18aabf42907d8cd95"
server_key_iv: "071fe3debba60cb0a605f85f3c764afd"
server_key_version: "encryptionkey@0"
I20260806 00:58:28.135564 24599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:28.140585 24599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:28.144402 24599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:28.144663 24599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:28.153472 24599 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42793
I20260806 00:58:28.153488 24683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42793 every 8 connection(s)
I20260806 00:58:28.154553 24599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:28.157593 24684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:28.160266 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 24599
I20260806 00:58:28.160444 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:28.160772 27959 external_mini_cluster.cc:1428] Setting key c5fbe14dd7075edba081de032df2e7bf
I20260806 00:58:28.164189 24684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a: Bootstrap starting.
I20260806 00:58:28.166687 24690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:28.162389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:35591 (local address 127.0.0.1:42793)
0806 00:58:28.162899 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:28.162915 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:28.162958 (+ 43us) server_negotiation.cc:407] Connection header received
0806 00:58:28.163768 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:28.163790 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:28.164110 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:28.164495 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:28.165391 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:28.165396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:28.165409 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:28.165420 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:28.165477 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:28.165497 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:28.165661 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:28.165754 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 00:58:28.165937 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":162,"threads_started":1}
I20260806 00:58:28.166787 24684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:28.167614 24684 log.cc:824] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:28.169932 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:28.170068 24684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a: No bootstrap required, opened a new log
I20260806 00:58:28.174409 24690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:28.172308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.157:40323 (local address 127.0.0.1:42793)
0806 00:58:28.172454 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:28.172460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:28.172706 (+ 246us) server_negotiation.cc:407] Connection header received
0806 00:58:28.172884 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:28.172888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:28.172932 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:28.173033 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:28.173852 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:28.173856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:28.173859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:28.173873 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:28.173908 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:28.173920 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:28.174020 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:28.174151 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 00:58:28.174199 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:58:28.174207 24684 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe93d5761f449238285796369322a6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42793 } }
I20260806 00:58:28.174634 24684 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:28.174737 24684 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebe93d5761f449238285796369322a6a, State: Initialized, Role: FOLLOWER
I20260806 00:58:28.175271 24684 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe93d5761f449238285796369322a6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42793 } }
I20260806 00:58:28.175449 24684 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:28.175546 24684 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:28.175673 24684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:28.176667 24684 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe93d5761f449238285796369322a6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42793 } }
I20260806 00:58:28.176990 24684 leader_election.cc:302] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebe93d5761f449238285796369322a6a; no voters:
I20260806 00:58:28.177278 24684 leader_election.cc:288] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:28.177423 24699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:28.177784 24699 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [term 1 LEADER]: Becoming Leader. State: Replica: ebe93d5761f449238285796369322a6a, State: Running, Role: LEADER
I20260806 00:58:28.178113 24699 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe93d5761f449238285796369322a6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42793 } }
I20260806 00:58:28.178583 24684 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:28.179869 24701 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebe93d5761f449238285796369322a6a. Latest consensus state: current_term: 1 leader_uuid: "ebe93d5761f449238285796369322a6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe93d5761f449238285796369322a6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42793 } } }
I20260806 00:58:28.180011 24701 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:28.180333 24700 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebe93d5761f449238285796369322a6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe93d5761f449238285796369322a6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42793 } } }
I20260806 00:58:28.180431 24700 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:28.180730 24712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:28.184773 24712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:28.190939 24712 catalog_manager.cc:1347] Generated new cluster ID: 6bb009890e5048c68f3a9d9bac4671c0
I20260806 00:58:28.191039 24712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:28.216109 24712 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:28.216476 24712 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:28.223378 24712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebe93d5761f449238285796369322a6a: Generated new TSK 0
I20260806 00:58:28.224252 24712 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:28.237035 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 24599
2026-08-06T00:58:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1681 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:28 dist-test-slave-sb2z krb5kdc[24729](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:28 dist-test-slave-sb2z krb5kdc[24729](info): set up 2 sockets
Aug 06 00:58:28 dist-test-slave-sb2z krb5kdc[24729](info): commencing operation
krb5kdc: starting...
W20260806 00:58:29.472087 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:29 dist-test-slave-sb2z krb5kdc[24729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:58:29.616374 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38087
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:36325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:38087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:29.732839 24744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:29.733196 24744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:29.733289 24744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:29.737632 24744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:29.737763 24744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:29.737804 24744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:29.737839 24744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:29.737879 24744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:29.743752 24744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:36325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:38087
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:38087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:29.745055 24744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:29.746191 24744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:29.755640 24750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:29.755668 24749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:29.755640 24754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:29.755992 24744 server_base.cc:1034] running on GCE node
I20260806 00:58:29.756587 24744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:29.757788 24744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:29.758980 24744 hybrid_clock.cc:648] HybridClock initialized: now 1785977909758940 us; error 54 us; skew 500 ppm
Aug 06 00:58:29 dist-test-slave-sb2z krb5kdc[24729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977909, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:29.762120 24744 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:58:29.763226 24744 webserver.cc:460] Webserver started at http://127.27.77.254:35955/ using document root <none> and password file <none>
I20260806 00:58:29.763944 24744 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:29.764045 24744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:29.764279 24744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:29.766433 24744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "3d44aa093ac949a29349f5260ae28138"
format_stamp: "Formatted at 2026-08-06 00:58:29 on dist-test-slave-sb2z"
server_key: "070d0ec5c76b789f70d2a8431a6b8bed"
server_key_iv: "881fa6ace41128654533475dbeeb89cf"
server_key_version: "encryptionkey@0"
I20260806 00:58:29.767030 24744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "3d44aa093ac949a29349f5260ae28138"
format_stamp: "Formatted at 2026-08-06 00:58:29 on dist-test-slave-sb2z"
server_key: "070d0ec5c76b789f70d2a8431a6b8bed"
server_key_iv: "881fa6ace41128654533475dbeeb89cf"
server_key_version: "encryptionkey@0"
I20260806 00:58:29.772254 24744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 00:58:29.775209 24764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:29.776623 24744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:29.776777 24744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "3d44aa093ac949a29349f5260ae28138"
format_stamp: "Formatted at 2026-08-06 00:58:29 on dist-test-slave-sb2z"
server_key: "070d0ec5c76b789f70d2a8431a6b8bed"
server_key_iv: "881fa6ace41128654533475dbeeb89cf"
server_key_version: "encryptionkey@0"
I20260806 00:58:29.776921 24744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:29.805382 24744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:29.809243 24744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:29.809497 24744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:29.817415 24744 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:38087
I20260806 00:58:29.817443 24826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:38087 every 8 connection(s)
I20260806 00:58:29.818473 24744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:29.821718 24827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:29.824927 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 24744
I20260806 00:58:29.825100 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:29.825457 27959 external_mini_cluster.cc:1428] Setting key 2d2724efed4152b55af882693041a1c7
I20260806 00:58:29.828905 24827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138: Bootstrap starting.
I20260806 00:58:29.831486 24827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:29.832377 24827 log.cc:824] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:29.834625 24827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138: No bootstrap required, opened a new log
Aug 06 00:58:29 dist-test-slave-sb2z krb5kdc[24729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:58:29.838294 24827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d44aa093ac949a29349f5260ae28138" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38087 } }
I20260806 00:58:29.838629 24827 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:29.838703 24827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d44aa093ac949a29349f5260ae28138, State: Initialized, Role: FOLLOWER
I20260806 00:58:29.839269 24827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d44aa093ac949a29349f5260ae28138" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38087 } }
I20260806 00:58:29.839491 24827 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:29.839607 24827 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:29.839759 24827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:29.841109 24827 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d44aa093ac949a29349f5260ae28138" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38087 } }
I20260806 00:58:29.841526 24827 leader_election.cc:302] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d44aa093ac949a29349f5260ae28138; no voters:
I20260806 00:58:29.841882 24827 leader_election.cc:288] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:29.842022 24834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:29.842283 24834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [term 1 LEADER]: Becoming Leader. State: Replica: 3d44aa093ac949a29349f5260ae28138, State: Running, Role: LEADER
I20260806 00:58:29.842512 24832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:29.827095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60753 (local address 127.27.77.254:38087)
0806 00:58:29.827680 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:29.827697 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:29.827737 (+ 40us) server_negotiation.cc:407] Connection header received
0806 00:58:29.828380 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:29.828398 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:29.828767 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:29.829167 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:29.830080 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:29.831109 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:29.832175 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:29.832719 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:29.832893 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:29.835961 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:29.835966 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:29.835980 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:29.836010 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:29.838423 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:29.839141 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:29.839147 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:29.839151 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:29.839283 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:29.839726 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:29.839729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:29.839731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:29.840297 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:29.840484 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:29.840975 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 00:58:29.841325 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":163,"threads_started":1}
I20260806 00:58:29.842657 24834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d44aa093ac949a29349f5260ae28138" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38087 } }
I20260806 00:58:29.843389 24827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:29.844794 24836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d44aa093ac949a29349f5260ae28138" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d44aa093ac949a29349f5260ae28138" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38087 } } }
I20260806 00:58:29.844941 24836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:29.845304 24837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d44aa093ac949a29349f5260ae28138. Latest consensus state: current_term: 1 leader_uuid: "3d44aa093ac949a29349f5260ae28138" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d44aa093ac949a29349f5260ae28138" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 38087 } } }
I20260806 00:58:29.845408 24837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:29.846155 24844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:29.846868 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:29.849787 24844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:29.857057 24844 catalog_manager.cc:1347] Generated new cluster ID: 073603d6c9bc48dc95d2448e0b8afc54
I20260806 00:58:29.857139 24844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:29.864913 24832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:29.852186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46167 (local address 127.27.77.254:38087)
0806 00:58:29.852338 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:29.852344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:29.854880 (+ 2536us) server_negotiation.cc:407] Connection header received
0806 00:58:29.855343 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:29.855348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:29.855405 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:29.855495 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:29.856566 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:29.857436 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:29.858644 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:29.859006 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:29.859140 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:29.860908 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:29.860927 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:29.860932 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:29.860962 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:29.862788 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:29.863404 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:29.863408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:29.863410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:29.863514 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:29.863922 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:29.863926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:29.863928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:29.864179 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:29.864348 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:29.864612 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 00:58:29.864731 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:58:29.880710 24844 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:29.881815 24844 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:29.887403 24844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d44aa093ac949a29349f5260ae28138: Generated new TSK 0
I20260806 00:58:29.888233 24844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:29.894737 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 24744
W20260806 00:58:29.905864 24727 connection.cc:535] client connection to 127.27.77.254:38087 recv error: Network error: failed to read from TLS socket (remote: 127.27.77.254:38087): Connection reset by peer (error 104)
2026-08-06T00:58:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1658 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:29 dist-test-slave-sb2z krb5kdc[24880](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:29 dist-test-slave-sb2z krb5kdc[24880](info): set up 2 sockets
Aug 06 00:58:29 dist-test-slave-sb2z krb5kdc[24880](info): commencing operation
krb5kdc: starting...
W20260806 00:58:31.108484 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:31 dist-test-slave-sb2z krb5kdc[24880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:58:31.248605 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:46289
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:46289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:31.370771 24895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:31.371089 24895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:31.371136 24895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:31.375375 24895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:31.375505 24895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:31.375546 24895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:31.375597 24895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:31.375653 24895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:31.381271 24895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:46289
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:46289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:31.382457 24895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:31.383611 24895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:31.393139 24904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:31.393265 24895 server_base.cc:1034] running on GCE node
W20260806 00:58:31.393180 24902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:31.393205 24900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:31.394040 24895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:31.395260 24895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:31.396490 24895 hybrid_clock.cc:648] HybridClock initialized: now 1785977911396460 us; error 43 us; skew 500 ppm
Aug 06 00:58:31 dist-test-slave-sb2z krb5kdc[24880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977911, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:31.399367 24895 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:58:31.400350 24895 webserver.cc:460] Webserver started at http://127.27.77.254:45083/ using document root <none> and password file <none>
I20260806 00:58:31.400995 24895 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:31.401098 24895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:31.401345 24895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:31.403491 24895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "acf5cc0360424b48aae432fb1c303782"
format_stamp: "Formatted at 2026-08-06 00:58:31 on dist-test-slave-sb2z"
server_key: "2c0d89fa875127d7b544ad153f619732"
server_key_iv: "2fa3c7b298de176e260643982ed04f3c"
server_key_version: "encryptionkey@0"
I20260806 00:58:31.404175 24895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "acf5cc0360424b48aae432fb1c303782"
format_stamp: "Formatted at 2026-08-06 00:58:31 on dist-test-slave-sb2z"
server_key: "2c0d89fa875127d7b544ad153f619732"
server_key_iv: "2fa3c7b298de176e260643982ed04f3c"
server_key_version: "encryptionkey@0"
I20260806 00:58:31.408166 24895 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:58:31.410800 24912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:31.412230 24895 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:31.412380 24895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "acf5cc0360424b48aae432fb1c303782"
format_stamp: "Formatted at 2026-08-06 00:58:31 on dist-test-slave-sb2z"
server_key: "2c0d89fa875127d7b544ad153f619732"
server_key_iv: "2fa3c7b298de176e260643982ed04f3c"
server_key_version: "encryptionkey@0"
I20260806 00:58:31.412501 24895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:31.430661 24895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:31.434193 24895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:31.434433 24895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:31.442462 24895 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:46289
I20260806 00:58:31.442477 24984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:46289 every 8 connection(s)
I20260806 00:58:31.443504 24895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:31.446427 24985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:31.446769 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 24895
I20260806 00:58:31.446925 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:31.447297 27959 external_mini_cluster.cc:1428] Setting key 0627a3d0ad7b0dfd9f6e873f154bbd18
I20260806 00:58:31.454493 24985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782: Bootstrap starting.
I20260806 00:58:31.457388 24985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:31.457610 24990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:31.448836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43205 (local address 127.27.77.254:46289)
0806 00:58:31.449312 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:31.449327 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:31.449362 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:58:31.450230 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:31.450252 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:31.450613 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:31.450917 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:31.451367 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:31.452503 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:31.453887 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:31.454356 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:31.454477 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:31.455519 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:31.455577 (+ 58us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:31.455598 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:31.456026 (+ 428us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:31.456086 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:31.456111 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:31.456216 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:31.456535 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 00:58:31.456699 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":138,"threads_started":1}
I20260806 00:58:31.458285 24985 log.cc:824] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:31.460587 24985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782: No bootstrap required, opened a new log
I20260806 00:58:31.460811 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:31.466077 24985 raft_consensus.cc:348] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf5cc0360424b48aae432fb1c303782" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46289 } }
I20260806 00:58:31.466385 24985 raft_consensus.cc:374] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:31.466460 24985 raft_consensus.cc:729] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf5cc0360424b48aae432fb1c303782, State: Initialized, Role: FOLLOWER
I20260806 00:58:31.467070 24985 consensus_queue.cc:260] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf5cc0360424b48aae432fb1c303782" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46289 } }
I20260806 00:58:31.467222 24985 raft_consensus.cc:388] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:31.467299 24985 raft_consensus.cc:482] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:31.467466 24985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:31.468366 24990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:31.463324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42485 (local address 127.27.77.254:46289)
0806 00:58:31.463500 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:31.463505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:31.463728 (+ 223us) server_negotiation.cc:407] Connection header received
0806 00:58:31.463860 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:31.463862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:31.463907 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:31.463989 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:31.464352 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:31.464937 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:31.466175 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:31.466539 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:31.466648 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:31.467794 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:31.467798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:31.467800 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:31.467815 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:31.467844 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:31.467855 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:31.467942 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:31.468140 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 00:58:31.468203 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:58:31.468912 24985 raft_consensus.cc:504] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf5cc0360424b48aae432fb1c303782" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46289 } }
I20260806 00:58:31.469336 24985 leader_election.cc:302] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acf5cc0360424b48aae432fb1c303782; no voters:
I20260806 00:58:31.469758 24985 leader_election.cc:288] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:31.469837 24997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:31.470156 24997 raft_consensus.cc:686] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [term 1 LEADER]: Becoming Leader. State: Replica: acf5cc0360424b48aae432fb1c303782, State: Running, Role: LEADER
I20260806 00:58:31.470502 24997 consensus_queue.cc:237] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf5cc0360424b48aae432fb1c303782" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46289 } }
I20260806 00:58:31.471408 24985 sys_catalog.cc:564] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:31.472493 24998 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acf5cc0360424b48aae432fb1c303782" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf5cc0360424b48aae432fb1c303782" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46289 } } }
I20260806 00:58:31.472551 24999 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acf5cc0360424b48aae432fb1c303782. Latest consensus state: current_term: 1 leader_uuid: "acf5cc0360424b48aae432fb1c303782" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf5cc0360424b48aae432fb1c303782" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46289 } } }
I20260806 00:58:31.472640 24998 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:31.472687 24999 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:31.473059 25006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:31.476747 25006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:31.482932 25006 catalog_manager.cc:1347] Generated new cluster ID: 5dd1f5fa0f6544b1b164a8a6f92d43a2
I20260806 00:58:31.483016 25006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:31.502735 25006 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:31.504259 25006 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:31.510977 25006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acf5cc0360424b48aae432fb1c303782: Generated new TSK 0
I20260806 00:58:31.511961 25006 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:31.520150 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 24895
W20260806 00:58:31.533872 24875 connection.cc:535] client connection to 127.27.77.254:46289 recv error: Network error: failed to read from TLS socket (remote: 127.27.77.254:46289): Connection reset by peer (error 104)
2026-08-06T00:58:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1626 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:31 dist-test-slave-sb2z krb5kdc[25026](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:31 dist-test-slave-sb2z krb5kdc[25026](info): set up 2 sockets
Aug 06 00:58:31 dist-test-slave-sb2z krb5kdc[25026](info): commencing operation
krb5kdc: starting...
W20260806 00:58:32.769315 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:32 dist-test-slave-sb2z krb5kdc[25026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:58:32.914827 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:39107
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:40627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:39107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:33.039649 25045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:33.040045 25045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:33.040138 25045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:33.044365 25045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:33.044474 25045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:33.044513 25045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:33.044553 25045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:33.044589 25045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:33.050102 25045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:40627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:39107
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:39107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:33.051360 25045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:33.052500 25045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:33.061743 25054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:33.061743 25052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:33.061753 25051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:33.062207 25045 server_base.cc:1034] running on GCE node
I20260806 00:58:33.062775 25045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:33.064030 25045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:33.065209 25045 hybrid_clock.cc:648] HybridClock initialized: now 1785977913065194 us; error 31 us; skew 500 ppm
Aug 06 00:58:33 dist-test-slave-sb2z krb5kdc[25026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977913, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:33.068548 25045 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:58:33.069605 25045 webserver.cc:460] Webserver started at http://127.27.77.254:46347/ using document root <none> and password file <none>
I20260806 00:58:33.070262 25045 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:33.070366 25045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:33.070621 25045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:33.072835 25045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "874033da3b8f47ad8a3ada8ad708bbf1"
format_stamp: "Formatted at 2026-08-06 00:58:33 on dist-test-slave-sb2z"
server_key: "e40f6456d6bb33ee124b40067d7e695d"
server_key_iv: "fd556d48bd21347b97dff552a27de568"
server_key_version: "encryptionkey@0"
I20260806 00:58:33.073446 25045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "874033da3b8f47ad8a3ada8ad708bbf1"
format_stamp: "Formatted at 2026-08-06 00:58:33 on dist-test-slave-sb2z"
server_key: "e40f6456d6bb33ee124b40067d7e695d"
server_key_iv: "fd556d48bd21347b97dff552a27de568"
server_key_version: "encryptionkey@0"
I20260806 00:58:33.077423 25045 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:58:33.080040 25062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:33.081389 25045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:33.081561 25045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "874033da3b8f47ad8a3ada8ad708bbf1"
format_stamp: "Formatted at 2026-08-06 00:58:33 on dist-test-slave-sb2z"
server_key: "e40f6456d6bb33ee124b40067d7e695d"
server_key_iv: "fd556d48bd21347b97dff552a27de568"
server_key_version: "encryptionkey@0"
I20260806 00:58:33.081684 25045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:33.086462 25045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:33.089708 25045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:33.089917 25045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:33.097613 25045 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:39107
I20260806 00:58:33.097643 25125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:39107 every 8 connection(s)
I20260806 00:58:33.098603 25045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:33.101472 25127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:33.103499 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 25045
I20260806 00:58:33.103694 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:33.104045 27959 external_mini_cluster.cc:1428] Setting key ce254e7cfc9119c438616a2c57544377
I20260806 00:58:33.108409 25127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1: Bootstrap starting.
I20260806 00:58:33.110339 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:33.105589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54093 (local address 127.27.77.254:39107)
0806 00:58:33.106054 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:33.106074 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:33.106119 (+ 45us) server_negotiation.cc:407] Connection header received
0806 00:58:33.107192 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:33.107217 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:33.107626 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:33.108002 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:33.108870 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:33.108876 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:33.108892 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:33.108904 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:33.108971 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:33.108997 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:33.109143 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:33.109226 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 00:58:33.109377 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":121,"threads_started":1}
I20260806 00:58:33.111225 25127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:33.112219 25127 log.cc:824] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:33.113725 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:33.114622 25127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1: No bootstrap required, opened a new log
I20260806 00:58:33.117722 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:33.115883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35387 (local address 127.27.77.254:39107)
0806 00:58:33.116020 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:33.116025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:33.116299 (+ 274us) server_negotiation.cc:407] Connection header received
0806 00:58:33.116429 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:33.116432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:33.116465 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:33.116527 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:33.117300 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:33.117303 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:33.117305 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:33.117316 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:33.117342 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:33.117352 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:33.117424 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:33.117537 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 00:58:33.117572 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:58:33.119479 25127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874033da3b8f47ad8a3ada8ad708bbf1" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39107 } }
I20260806 00:58:33.119812 25127 raft_consensus.cc:374] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:33.119886 25127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874033da3b8f47ad8a3ada8ad708bbf1, State: Initialized, Role: FOLLOWER
I20260806 00:58:33.120355 25127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874033da3b8f47ad8a3ada8ad708bbf1" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39107 } }
I20260806 00:58:33.120528 25127 raft_consensus.cc:388] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:33.120602 25127 raft_consensus.cc:482] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:33.120704 25127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:33.121711 25127 raft_consensus.cc:504] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874033da3b8f47ad8a3ada8ad708bbf1" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39107 } }
I20260806 00:58:33.122097 25127 leader_election.cc:302] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 874033da3b8f47ad8a3ada8ad708bbf1; no voters:
I20260806 00:58:33.122442 25127 leader_election.cc:288] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:33.122614 25143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:33.122905 25143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [term 1 LEADER]: Becoming Leader. State: Replica: 874033da3b8f47ad8a3ada8ad708bbf1, State: Running, Role: LEADER
I20260806 00:58:33.123237 25143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874033da3b8f47ad8a3ada8ad708bbf1" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39107 } }
I20260806 00:58:33.123914 25127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:33.125244 25146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 874033da3b8f47ad8a3ada8ad708bbf1. Latest consensus state: current_term: 1 leader_uuid: "874033da3b8f47ad8a3ada8ad708bbf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874033da3b8f47ad8a3ada8ad708bbf1" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39107 } } }
I20260806 00:58:33.125380 25146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:33.125684 25144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "874033da3b8f47ad8a3ada8ad708bbf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874033da3b8f47ad8a3ada8ad708bbf1" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 39107 } } }
I20260806 00:58:33.125801 25144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:33.125869 25156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:33.128805 25156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:33.134709 25156 catalog_manager.cc:1347] Generated new cluster ID: 6bb8f4eb6e6d423eb61c9e9cdee734dc
I20260806 00:58:33.134800 25156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:33.156121 25156 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:33.156515 25156 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:33.162144 25156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 874033da3b8f47ad8a3ada8ad708bbf1: Generated new TSK 0
I20260806 00:58:33.162997 25156 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:33.174688 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 25045
2026-08-06T00:58:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1652 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:33 dist-test-slave-sb2z krb5kdc[25172](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:33 dist-test-slave-sb2z krb5kdc[25172](info): set up 2 sockets
Aug 06 00:58:33 dist-test-slave-sb2z krb5kdc[25172](info): commencing operation
krb5kdc: starting...
W20260806 00:58:34.439641 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:34 dist-test-slave-sb2z krb5kdc[25172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:58:34.579350 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:40729
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:46607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:40729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:34.696560 25187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:34.696892 25187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:34.696974 25187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:34.701121 25187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:34.701215 25187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:34.701253 25187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:34.701313 25187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:34.701354 25187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:34.706848 25187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:46607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:40729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:40729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:34.708015 25187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:34.709043 25187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:34.718180 25196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:34.718170 25195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:34.718166 25199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:34.718240 25187 server_base.cc:1034] running on GCE node
I20260806 00:58:34.719105 25187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:34.720341 25187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:34.721541 25187 hybrid_clock.cc:648] HybridClock initialized: now 1785977914721526 us; error 28 us; skew 500 ppm
Aug 06 00:58:34 dist-test-slave-sb2z krb5kdc[25172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977914, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:34.724594 25187 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:58:34.725615 25187 webserver.cc:460] Webserver started at http://127.27.77.254:42833/ using document root <none> and password file <none>
I20260806 00:58:34.726215 25187 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:34.726310 25187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:34.726550 25187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:34.728835 25187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "60aa5ca89e5849629234523b79955232"
format_stamp: "Formatted at 2026-08-06 00:58:34 on dist-test-slave-sb2z"
server_key: "07146b96b2b1efaaf47802b5b5adef1b"
server_key_iv: "d58b05770d65ff5084c7694b0db083cf"
server_key_version: "encryptionkey@0"
I20260806 00:58:34.729450 25187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "60aa5ca89e5849629234523b79955232"
format_stamp: "Formatted at 2026-08-06 00:58:34 on dist-test-slave-sb2z"
server_key: "07146b96b2b1efaaf47802b5b5adef1b"
server_key_iv: "d58b05770d65ff5084c7694b0db083cf"
server_key_version: "encryptionkey@0"
I20260806 00:58:34.733474 25187 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:58:34.736064 25206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:34.737448 25187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:34.737600 25187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "60aa5ca89e5849629234523b79955232"
format_stamp: "Formatted at 2026-08-06 00:58:34 on dist-test-slave-sb2z"
server_key: "07146b96b2b1efaaf47802b5b5adef1b"
server_key_iv: "d58b05770d65ff5084c7694b0db083cf"
server_key_version: "encryptionkey@0"
I20260806 00:58:34.737720 25187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:34.763008 25187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:34.766672 25187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:34.766904 25187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:34.774917 25187 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:40729
I20260806 00:58:34.774995 25273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:40729 every 8 connection(s)
I20260806 00:58:34.776044 25187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:34.777782 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 25187
I20260806 00:58:34.777940 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:34.778259 27959 external_mini_cluster.cc:1428] Setting key 2d3e41bc989bc580de52289f9f87c531
I20260806 00:58:34.779335 25274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:34.786352 25274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232: Bootstrap starting.
Aug 06 00:58:34 dist-test-slave-sb2z krb5kdc[25172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:58:34.789105 25274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:34.790206 25274 log.cc:824] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:34.793094 25274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232: No bootstrap required, opened a new log
I20260806 00:58:34.794857 25277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:34.779858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54727 (local address 127.27.77.254:40729)
0806 00:58:34.780362 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:34.780381 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:34.780424 (+ 43us) server_negotiation.cc:407] Connection header received
0806 00:58:34.781311 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:34.781337 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:34.781769 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:34.782154 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:34.782875 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:34.784405 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:34.785306 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:34.785976 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:34.786157 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:34.789019 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:34.789029 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:34.789052 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:34.789098 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:34.791362 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:34.791945 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:34.791952 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:34.791958 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:34.792307 (+ 349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:34.792735 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:34.792739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:34.792741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:34.793235 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:34.793438 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:34.793767 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 00:58:34.794121 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":147,"threads_started":1}
I20260806 00:58:34.797394 25274 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60aa5ca89e5849629234523b79955232" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40729 } }
I20260806 00:58:34.797678 25274 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:34.797744 25274 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60aa5ca89e5849629234523b79955232, State: Initialized, Role: FOLLOWER
I20260806 00:58:34.798141 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:34.798226 25274 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60aa5ca89e5849629234523b79955232" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40729 } }
I20260806 00:58:34.798381 25274 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:34.798446 25274 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:34.798532 25274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:34.799966 25274 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60aa5ca89e5849629234523b79955232" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40729 } }
I20260806 00:58:34.800384 25274 leader_election.cc:302] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60aa5ca89e5849629234523b79955232; no voters:
I20260806 00:58:34.800773 25274 leader_election.cc:288] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:34.800980 25285 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:34.801290 25285 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [term 1 LEADER]: Becoming Leader. State: Replica: 60aa5ca89e5849629234523b79955232, State: Running, Role: LEADER
I20260806 00:58:34.801606 25285 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60aa5ca89e5849629234523b79955232" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40729 } }
I20260806 00:58:34.802094 25274 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:34.803390 25289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60aa5ca89e5849629234523b79955232. Latest consensus state: current_term: 1 leader_uuid: "60aa5ca89e5849629234523b79955232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60aa5ca89e5849629234523b79955232" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40729 } } }
I20260806 00:58:34.803522 25289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:34.804039 25288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60aa5ca89e5849629234523b79955232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60aa5ca89e5849629234523b79955232" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 40729 } } }
I20260806 00:58:34.804203 25288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232 [sys.catalog]: This master's current role is: LEADER
W20260806 00:58:34.808461 25304 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:58:34.808552 25304 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:58:34.808670 25297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:34.809834 25297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:34.815996 25297 catalog_manager.cc:1347] Generated new cluster ID: e3871b8a72ed42249bea028beb9bfaba
I20260806 00:58:34.816085 25297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:34.830541 25297 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:34.831647 25297 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:34.841149 25297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60aa5ca89e5849629234523b79955232: Generated new TSK 0
I20260806 00:58:34.842013 25297 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:58:35.786268 25286 sasl_common.cc:323] Time spent creating new SASL kudu client: real 0.979s user 0.000s sys 0.000s
I20260806 00:58:35.790937 25277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:34.800776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60079 (local address 127.27.77.254:40729)
0806 00:58:34.800908 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:34.800927 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:34.803154 (+ 2227us) server_negotiation.cc:407] Connection header received
0806 00:58:34.803368 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:34.803372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:34.803427 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:34.803486 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:34.804541 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:34.805373 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:34.806540 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:34.806922 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:34.807062 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:35.787392 (+980330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:35.787396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:35.787399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:35.787436 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:35.789199 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:35.789677 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:35.789679 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:35.789681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:35.789764 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:35.790017 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:35.790019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:35.790020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:35.790249 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:35.790353 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:35.790583 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 00:58:35.790715 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:58:35.794786 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 25187
2026-08-06T00:58:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2622 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:35 dist-test-slave-sb2z krb5kdc[25313](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:35 dist-test-slave-sb2z krb5kdc[25313](info): set up 2 sockets
Aug 06 00:58:35 dist-test-slave-sb2z krb5kdc[25313](info): commencing operation
krb5kdc: starting...
W20260806 00:58:37.098846 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:37 dist-test-slave-sb2z krb5kdc[25313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:58:37.236369 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:44085
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:44085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:37.354827 25332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:37.355132 25332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:37.355176 25332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:37.359261 25332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:37.359344 25332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:37.359364 25332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:37.359380 25332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:37.359395 25332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:37.365046 25332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:44085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:44085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:37.366202 25332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:37.367223 25332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:37.376469 25341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:37.376520 25338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:37.376528 25337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:37.376621 25332 server_base.cc:1034] running on GCE node
I20260806 00:58:37.377343 25332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:37.378531 25332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:37.379748 25332 hybrid_clock.cc:648] HybridClock initialized: now 1785977917379727 us; error 35 us; skew 500 ppm
Aug 06 00:58:37 dist-test-slave-sb2z krb5kdc[25313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977917, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:37.382642 25332 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:58:37.383512 25332 webserver.cc:460] Webserver started at http://127.27.77.254:37429/ using document root <none> and password file <none>
I20260806 00:58:37.384130 25332 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:37.384191 25332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:37.384359 25332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:37.386407 25332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "d2e19bf10ac04e3bacfca8543ddc4f01"
format_stamp: "Formatted at 2026-08-06 00:58:37 on dist-test-slave-sb2z"
server_key: "19b675018201b1d6cf15243a6d50e56b"
server_key_iv: "f736ecca7448c3602e79741cc4f34e33"
server_key_version: "encryptionkey@0"
I20260806 00:58:37.386955 25332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "d2e19bf10ac04e3bacfca8543ddc4f01"
format_stamp: "Formatted at 2026-08-06 00:58:37 on dist-test-slave-sb2z"
server_key: "19b675018201b1d6cf15243a6d50e56b"
server_key_iv: "f736ecca7448c3602e79741cc4f34e33"
server_key_version: "encryptionkey@0"
I20260806 00:58:37.390743 25332 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 00:58:37.393252 25349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:37.394565 25332 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:58:37.394743 25332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "d2e19bf10ac04e3bacfca8543ddc4f01"
format_stamp: "Formatted at 2026-08-06 00:58:37 on dist-test-slave-sb2z"
server_key: "19b675018201b1d6cf15243a6d50e56b"
server_key_iv: "f736ecca7448c3602e79741cc4f34e33"
server_key_version: "encryptionkey@0"
I20260806 00:58:37.394851 25332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:37.414410 25332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:37.417948 25332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:37.418180 25332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:37.426994 25332 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:44085
I20260806 00:58:37.427188 25412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:44085 every 8 connection(s)
I20260806 00:58:37.428398 25332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:37.432190 25413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:37.434454 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 25332
I20260806 00:58:37.434626 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:37.434958 27959 external_mini_cluster.cc:1428] Setting key 339c5f2ba82b9bfce53f0e10477acf41
I20260806 00:58:37.440711 25413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01: Bootstrap starting.
I20260806 00:58:37.444020 25413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:37.444176 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:37.436334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53883 (local address 127.27.77.254:44085)
0806 00:58:37.436787 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:37.436803 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:37.436839 (+ 36us) server_negotiation.cc:407] Connection header received
0806 00:58:37.437493 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:37.437511 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:37.437856 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:37.438209 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:37.438604 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:37.439692 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:37.440627 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:37.441186 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:37.441372 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:37.442269 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:37.442275 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:37.442287 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:37.442675 (+ 388us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:37.442731 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:37.442755 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:37.442864 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:37.443028 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 00:58:37.443273 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":139,"threads_started":1}
I20260806 00:58:37.445043 25413 log.cc:824] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:37.447260 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:37.447556 25413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01: No bootstrap required, opened a new log
I20260806 00:58:37.452719 25413 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e19bf10ac04e3bacfca8543ddc4f01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 44085 } }
I20260806 00:58:37.453199 25413 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:37.453322 25413 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e19bf10ac04e3bacfca8543ddc4f01, State: Initialized, Role: FOLLOWER
I20260806 00:58:37.453951 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:37.449426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54989 (local address 127.27.77.254:44085)
0806 00:58:37.449554 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:37.449558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:37.449926 (+ 368us) server_negotiation.cc:407] Connection header received
0806 00:58:37.450049 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:37.450052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:37.450094 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:37.450186 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:37.450456 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:37.451319 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:37.452277 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:37.452514 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:37.452628 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:37.453481 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:37.453484 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:37.453487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:37.453502 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:37.453534 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:37.453544 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:37.453625 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:37.453758 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 00:58:37.453815 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:58:37.453926 25413 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e19bf10ac04e3bacfca8543ddc4f01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 44085 } }
I20260806 00:58:37.454105 25413 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:37.454210 25413 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:37.454347 25413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:37.455818 25413 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e19bf10ac04e3bacfca8543ddc4f01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 44085 } }
I20260806 00:58:37.456254 25413 leader_election.cc:302] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2e19bf10ac04e3bacfca8543ddc4f01; no voters:
I20260806 00:58:37.456663 25413 leader_election.cc:288] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:37.456965 25423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:37.457280 25423 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [term 1 LEADER]: Becoming Leader. State: Replica: d2e19bf10ac04e3bacfca8543ddc4f01, State: Running, Role: LEADER
I20260806 00:58:37.457628 25423 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e19bf10ac04e3bacfca8543ddc4f01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 44085 } }
I20260806 00:58:37.458231 25413 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:37.459511 25425 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2e19bf10ac04e3bacfca8543ddc4f01. Latest consensus state: current_term: 1 leader_uuid: "d2e19bf10ac04e3bacfca8543ddc4f01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e19bf10ac04e3bacfca8543ddc4f01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 44085 } } }
I20260806 00:58:37.459578 25424 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2e19bf10ac04e3bacfca8543ddc4f01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e19bf10ac04e3bacfca8543ddc4f01" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 44085 } } }
I20260806 00:58:37.459694 25424 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:37.459694 25425 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:37.460286 25431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:37.464877 25431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:37.472131 25431 catalog_manager.cc:1347] Generated new cluster ID: 972ae2ae1e194d10bec74549bbd38dd1
I20260806 00:58:37.472312 25431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:37.486296 25431 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:37.487320 25431 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:37.493417 25431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2e19bf10ac04e3bacfca8543ddc4f01: Generated new TSK 0
I20260806 00:58:37.494128 25431 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:37.501147 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 25332
2026-08-06T00:58:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1708 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:37 dist-test-slave-sb2z krb5kdc[25453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:37 dist-test-slave-sb2z krb5kdc[25453](info): set up 2 sockets
Aug 06 00:58:37 dist-test-slave-sb2z krb5kdc[25453](info): commencing operation
krb5kdc: starting...
W20260806 00:58:38.712934 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:38 dist-test-slave-sb2z krb5kdc[25453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:58:38.859362 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:37513
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:37513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:38.985157 25468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:38.985613 25468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:38.985730 25468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:38.990057 25468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:38.990180 25468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:38.990226 25468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:38.990315 25468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:38.990371 25468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:38.996647 25468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:37513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:37513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:38.997965 25468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:38.999128 25468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:39.009505 25477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:39.009615 25480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:39.009809 25468 server_base.cc:1034] running on GCE node
W20260806 00:58:39.009615 25476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:39.010672 25468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:39.012147 25468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:39.013418 25468 hybrid_clock.cc:648] HybridClock initialized: now 1785977919013381 us; error 54 us; skew 500 ppm
Aug 06 00:58:39 dist-test-slave-sb2z krb5kdc[25453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977919, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:39.017100 25468 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:58:39.018199 25468 webserver.cc:460] Webserver started at http://127.27.77.254:45139/ using document root <none> and password file <none>
I20260806 00:58:39.018822 25468 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:39.018924 25468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:39.019187 25468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:39.021593 25468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "8a31d43dc9d84752a29d0bcf9dd151f8"
format_stamp: "Formatted at 2026-08-06 00:58:39 on dist-test-slave-sb2z"
server_key: "745f30cd250648cd4dac9568e82e5096"
server_key_iv: "fb4a90d53b393db152769c2d55e00994"
server_key_version: "encryptionkey@0"
I20260806 00:58:39.022233 25468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "8a31d43dc9d84752a29d0bcf9dd151f8"
format_stamp: "Formatted at 2026-08-06 00:58:39 on dist-test-slave-sb2z"
server_key: "745f30cd250648cd4dac9568e82e5096"
server_key_iv: "fb4a90d53b393db152769c2d55e00994"
server_key_version: "encryptionkey@0"
I20260806 00:58:39.026443 25468 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 00:58:39.029349 25491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:39.030886 25468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 00:58:39.031069 25468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "8a31d43dc9d84752a29d0bcf9dd151f8"
format_stamp: "Formatted at 2026-08-06 00:58:39 on dist-test-slave-sb2z"
server_key: "745f30cd250648cd4dac9568e82e5096"
server_key_iv: "fb4a90d53b393db152769c2d55e00994"
server_key_version: "encryptionkey@0"
I20260806 00:58:39.031209 25468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:39.036712 25468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:39.040486 25468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:39.040762 25468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:39.049211 25468 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:37513
I20260806 00:58:39.049245 25550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:37513 every 8 connection(s)
I20260806 00:58:39.050257 25468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:39.053407 25552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:39.058372 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 25468
I20260806 00:58:39.058696 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:39.059149 27959 external_mini_cluster.cc:1428] Setting key 5e751ae70f2c62e76786bf42c2047abc
I20260806 00:58:39.060513 25552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8: Bootstrap starting.
I20260806 00:58:39.063110 25552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:39.064177 25552 log.cc:824] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:39.065284 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:39.060841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41399 (local address 127.27.77.254:37513)
0806 00:58:39.061302 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:39.061320 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:39.061357 (+ 37us) server_negotiation.cc:407] Connection header received
0806 00:58:39.062051 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:39.062070 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:39.062360 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:39.062652 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:39.063694 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:39.063699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:39.063712 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:39.063724 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:39.063783 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:39.063803 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:39.063910 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:39.064184 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 00:58:39.064372 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":146,"threads_started":1}
I20260806 00:58:39.066517 25552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8: No bootstrap required, opened a new log
I20260806 00:58:39.068348 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:39.071239 25552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a31d43dc9d84752a29d0bcf9dd151f8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37513 } }
I20260806 00:58:39.071587 25552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:58:39.071659 25552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a31d43dc9d84752a29d0bcf9dd151f8, State: Initialized, Role: FOLLOWER
I20260806 00:58:39.072167 25552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a31d43dc9d84752a29d0bcf9dd151f8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37513 } }
I20260806 00:58:39.072362 25552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:58:39.072448 25552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:58:39.072578 25552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:39.072808 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:39.070889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49647 (local address 127.27.77.254:37513)
0806 00:58:39.071022 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:39.071027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:39.071191 (+ 164us) server_negotiation.cc:407] Connection header received
0806 00:58:39.071286 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:39.071289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:39.071327 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:39.071422 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:39.072252 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:39.072255 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:39.072258 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:58:39.072271 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:39.072303 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:58:39.072318 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:39.072418 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:39.072625 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 00:58:39.072675 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:58:39.073710 25552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a31d43dc9d84752a29d0bcf9dd151f8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37513 } }
I20260806 00:58:39.074074 25552 leader_election.cc:302] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a31d43dc9d84752a29d0bcf9dd151f8; no voters:
I20260806 00:58:39.074398 25552 leader_election.cc:288] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:58:39.074508 25564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:39.074756 25564 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [term 1 LEADER]: Becoming Leader. State: Replica: 8a31d43dc9d84752a29d0bcf9dd151f8, State: Running, Role: LEADER
I20260806 00:58:39.075134 25564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a31d43dc9d84752a29d0bcf9dd151f8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37513 } }
I20260806 00:58:39.075773 25552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:39.076865 25565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a31d43dc9d84752a29d0bcf9dd151f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a31d43dc9d84752a29d0bcf9dd151f8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37513 } } }
I20260806 00:58:39.076996 25565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:39.077308 25569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a31d43dc9d84752a29d0bcf9dd151f8. Latest consensus state: current_term: 1 leader_uuid: "8a31d43dc9d84752a29d0bcf9dd151f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a31d43dc9d84752a29d0bcf9dd151f8" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37513 } } }
I20260806 00:58:39.077407 25582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:39.077433 25569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:39.080685 25582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:39.086408 25582 catalog_manager.cc:1347] Generated new cluster ID: 7905aa6a977b440187040a0480201918
I20260806 00:58:39.086488 25582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:39.109936 25582 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:39.110332 25582 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:39.122941 25582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a31d43dc9d84752a29d0bcf9dd151f8: Generated new TSK 0
I20260806 00:58:39.123859 25582 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:58:39.129990 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 25468
2026-08-06T00:58:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1625 ms)
[----------] 18 tests from AuthTokenIssuingTest (30657 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:39 dist-test-slave-sb2z krb5kdc[25597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:39 dist-test-slave-sb2z krb5kdc[25597](info): set up 2 sockets
Aug 06 00:58:39 dist-test-slave-sb2z krb5kdc[25597](info): commencing operation
krb5kdc: starting...
W20260806 00:58:40.341775 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:40 dist-test-slave-sb2z krb5kdc[25597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:58:40.492327 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:37179
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:35367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:37179,127.27.77.253:34599,127.27.77.252:36691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:40.622725 25610 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:58:40.623263 25610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:40.623327 25610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:40.623418 25610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:40.629772 25610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:40.629853 25610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:40.629873 25610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:40.629891 25610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:40.629906 25610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:40.635582 25610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:35367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:37179,127.27.77.253:34599,127.27.77.252:36691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:37179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:40.636754 25610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:40.637813 25610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:40.647308 25616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:40.647332 25619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:40.647374 25610 server_base.cc:1034] running on GCE node
W20260806 00:58:40.647315 25617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:40.648216 25610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:40.649403 25610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:40.650568 25610 hybrid_clock.cc:648] HybridClock initialized: now 1785977920650544 us; error 37 us; skew 500 ppm
Aug 06 00:58:40 dist-test-slave-sb2z krb5kdc[25597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977920, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:40.653494 25610 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:58:40.654567 25610 webserver.cc:460] Webserver started at http://127.27.77.254:40113/ using document root <none> and password file <none>
I20260806 00:58:40.655108 25610 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:40.655167 25610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:40.655331 25610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:40.657487 25610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "45e455f7040043c6ac5bc03441cde531"
format_stamp: "Formatted at 2026-08-06 00:58:40 on dist-test-slave-sb2z"
server_key: "8cb3a3a375b9fdeb6d045e6f624713f1"
server_key_iv: "32eb475bda8e73f3a7343904d0754fa8"
server_key_version: "encryptionkey@0"
I20260806 00:58:40.658023 25610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "45e455f7040043c6ac5bc03441cde531"
format_stamp: "Formatted at 2026-08-06 00:58:40 on dist-test-slave-sb2z"
server_key: "8cb3a3a375b9fdeb6d045e6f624713f1"
server_key_iv: "32eb475bda8e73f3a7343904d0754fa8"
server_key_version: "encryptionkey@0"
I20260806 00:58:40.662005 25610 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 00:58:40.664654 25631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:40.666090 25610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:40.666266 25610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "45e455f7040043c6ac5bc03441cde531"
format_stamp: "Formatted at 2026-08-06 00:58:40 on dist-test-slave-sb2z"
server_key: "8cb3a3a375b9fdeb6d045e6f624713f1"
server_key_iv: "32eb475bda8e73f3a7343904d0754fa8"
server_key_version: "encryptionkey@0"
I20260806 00:58:40.666399 25610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:40.676859 25610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:40.680032 25610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:40.680272 25610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:40.688189 25610 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:37179
I20260806 00:58:40.688503 25702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:37179 every 8 connection(s)
I20260806 00:58:40.689270 25610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:40.691258 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 25610
I20260806 00:58:40.691402 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:40.691792 27959 external_mini_cluster.cc:1428] Setting key a69989895f93d7c1472e7445486d39db
I20260806 00:58:40.692761 25704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:40.698210 25704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.77.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.253@KRBTEST.COM" created.
Aug 06 00:58:40 dist-test-slave-sb2z krb5kdc[25597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977920, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:58:40.716209 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.700875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:37179 (local address 127.0.0.1:39394)
0806 00:58:40.701549 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:40.701588 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:40.702596 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:40.703388 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:40.703394 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:40.703914 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:40.704177 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:40.704192 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:40.705660 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:40.705668 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:40.706821 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:40.706837 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:40.707857 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:40.707865 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:40.708051 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:40.708868 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:40.708886 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:40.710413 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:40.712774 (+ 2361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:40.712786 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:40.712797 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:40.713040 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:40.713387 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:40.713392 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:40.713394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:40.713522 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:40.714160 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:40.714167 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:40.714316 (+ 149us) client_negotiation.cc:769] Sending connection context
0806 00:58:40.714506 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 00:58:40.714903 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":114,"spinlock_wait_cycles":23936,"thread_start_us":176,"threads_started":1}
I20260806 00:58:40.716209 25708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.700875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39394 (local address 127.27.77.254:37179)
0806 00:58:40.701385 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:40.701408 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:40.701712 (+ 304us) server_negotiation.cc:407] Connection header received
0806 00:58:40.702941 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:40.702945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:40.703142 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:40.703353 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:40.704351 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:40.705520 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:40.707022 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:40.707670 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:40.707816 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:40.710573 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:40.710582 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:40.710598 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:40.710628 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:40.712652 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:40.713159 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:40.713179 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:40.713180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:40.713287 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:40.713619 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:40.713623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:40.713625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:40.714032 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:40.714181 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:40.714563 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 00:58:40.714849 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":347,"thread_start_us":175,"threads_started":1}
I20260806 00:58:40.719281 25704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:58:40.720217 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.719863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:34599 (local address 127.0.0.1:41570)
0806 00:58:40.720024 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:40.720138 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:34599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260806 00:58:40.720444 25637 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.77.253:34599: connect: Connection refused (error 111)
I20260806 00:58:40.721480 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.721188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:34599 (local address 127.0.0.1:41574)
0806 00:58:40.721360 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:40.721419 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:34599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 00:58:40.721649 25704 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.253:34599: Network error: Client connection negotiation failed: client connection to 127.27.77.253:34599: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.253.keytab.
Entry for principal kudu/127.27.77.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.253.keytab.
I20260806 00:58:40.727869 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.253:34599
--webserver_interface=127.27.77.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.253.keytab
--principal=kudu/127.27.77.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:35367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:37179,127.27.77.253:34599,127.27.77.252:36691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:58:40.770915 25704 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } attempt: 1
I20260806 00:58:40.771929 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.771600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:34599 (local address 127.0.0.1:41586)
0806 00:58:40.771765 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:40.771833 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:34599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 00:58:40.772773 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.772539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:34599 (local address 127.0.0.1:41590)
0806 00:58:40.772660 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:40.772709 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:34599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 00:58:40.772959 25704 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.253:34599: Network error: Client connection negotiation failed: client connection to 127.27.77.253:34599: connect: Connection refused (error 111)
I20260806 00:58:40.841234 25704 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } attempt: 2
I20260806 00:58:40.842406 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.842074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:34599 (local address 127.0.0.1:41606)
0806 00:58:40.842254 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:40.842321 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:34599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 00:58:40.843312 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.843078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:34599 (local address 127.0.0.1:41610)
0806 00:58:40.843197 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:40.843251 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:34599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 00:58:40.843484 25704 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.253:34599: Network error: Client connection negotiation failed: client connection to 127.27.77.253:34599: connect: Connection refused (error 111)
W20260806 00:58:40.854833 25713 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:58:40.855185 25713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:40.855260 25713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:40.855340 25713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:40.859750 25713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:40.859833 25713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:40.859858 25713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:40.859875 25713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:40.859917 25713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:40.865645 25713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:35367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:37179,127.27.77.253:34599,127.27.77.252:36691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.253.keytab
--principal=kudu/127.27.77.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.253:34599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.77.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:40.866905 25713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:40.868039 25713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:40.877446 25725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:40.877450 25723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:40.877599 25720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:40.877904 25713 server_base.cc:1034] running on GCE node
I20260806 00:58:40.878793 25713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:40.880499 25713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:40.881734 25713 hybrid_clock.cc:648] HybridClock initialized: now 1785977920881645 us; error 106 us; skew 500 ppm
Aug 06 00:58:40 dist-test-slave-sb2z krb5kdc[25597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977920, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:40.888368 25713 init.cc:377] Logged in from keytab as kudu/127.27.77.253@KRBTEST.COM (short username kudu)
I20260806 00:58:40.889535 25713 webserver.cc:460] Webserver started at http://127.27.77.253:37041/ using document root <none> and password file <none>
I20260806 00:58:40.890180 25713 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:40.890249 25713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:40.890492 25713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:40.892899 25713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/data/instance:
uuid: "52e496d0f1df47909fadac5e3e7626df"
format_stamp: "Formatted at 2026-08-06 00:58:40 on dist-test-slave-sb2z"
server_key: "d60bfff3c8285d824cf05d3f1d32d8ad"
server_key_iv: "f139f146cafd83b0b5c2470150112f61"
server_key_version: "encryptionkey@0"
I20260806 00:58:40.893525 25713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/wal/instance:
uuid: "52e496d0f1df47909fadac5e3e7626df"
format_stamp: "Formatted at 2026-08-06 00:58:40 on dist-test-slave-sb2z"
server_key: "d60bfff3c8285d824cf05d3f1d32d8ad"
server_key_iv: "f139f146cafd83b0b5c2470150112f61"
server_key_version: "encryptionkey@0"
I20260806 00:58:40.897655 25713 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:58:40.900255 25733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:40.901716 25713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:58:40.901878 25713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/wal
uuid: "52e496d0f1df47909fadac5e3e7626df"
format_stamp: "Formatted at 2026-08-06 00:58:40 on dist-test-slave-sb2z"
server_key: "d60bfff3c8285d824cf05d3f1d32d8ad"
server_key_iv: "f139f146cafd83b0b5c2470150112f61"
server_key_version: "encryptionkey@0"
I20260806 00:58:40.902005 25713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:40.926980 25713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:40.930727 25713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:40.930966 25713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:40.934674 25704 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } attempt: 3
I20260806 00:58:40.935596 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.935201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:34599 (local address 127.0.0.1:41626)
0806 00:58:40.935403 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:40.935483 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:34599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 00:58:40.936524 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.936192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:34599 (local address 127.0.0.1:41634)
0806 00:58:40.936400 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:40.936458 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:34599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260806 00:58:40.936735 25704 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.253:34599: Network error: Client connection negotiation failed: client connection to 127.27.77.253:34599: connect: Connection refused (error 111)
I20260806 00:58:40.939294 25713 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.253:34599
I20260806 00:58:40.939666 25800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.253:34599 every 8 connection(s)
I20260806 00:58:40.940404 25713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/data/info.pb
I20260806 00:58:40.943503 25803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:40.946722 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 25713
I20260806 00:58:40.946913 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-1/wal/instance
I20260806 00:58:40.947288 27959 external_mini_cluster.cc:1428] Setting key fc21d5d9e20277a866da77153718f287
I20260806 00:58:40.948058 25803 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.77.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.252@KRBTEST.COM" created.
Aug 06 00:58:40 dist-test-slave-sb2z krb5kdc[25597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977920, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.253@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:58:40.967425 25708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.950212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39400 (local address 127.27.77.254:37179)
0806 00:58:40.950409 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:40.950414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:40.951477 (+ 1063us) server_negotiation.cc:407] Connection header received
0806 00:58:40.953123 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:40.953126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:40.953225 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:40.953339 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:40.955263 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:40.955892 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:40.957899 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:40.958183 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:40.958276 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:40.962923 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:40.962926 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:40.962928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:40.962958 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:40.964917 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:40.965542 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:40.965548 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:40.965553 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:40.965660 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:40.966018 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:40.966024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:40.966028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:40.966313 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:40.966422 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:40.967020 (+ 598us) server_negotiation.cc:299] Negotiation successful
0806 00:58:40.967192 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 00:58:40.968328 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.950651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:37179 (local address 127.0.0.1:39400)
0806 00:58:40.951260 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:40.951326 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:40.952826 (+ 1500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:40.953580 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:40.953592 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:40.954203 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:40.955060 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:40.955087 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:40.956059 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:40.956063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:40.957742 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:40.957758 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:40.958375 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:40.958380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:40.958626 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:40.959313 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:40.959332 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:40.962694 (+ 3362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:40.965101 (+ 2407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:40.965107 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:40.965121 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:40.965426 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:40.965763 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:40.965765 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:40.965767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:40.965915 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:40.966429 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:40.966435 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:40.966718 (+ 283us) client_negotiation.cc:769] Sending connection context
0806 00:58:40.966900 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 00:58:40.967240 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":133,"threads_started":1}
I20260806 00:58:40.969998 25803 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:58:40 dist-test-slave-sb2z krb5kdc[25597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977920, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.253@KRBTEST.COM for kudu/127.27.77.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.252.keytab.
Entry for principal kudu/127.27.77.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.252.keytab.
I20260806 00:58:40.980573 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.970618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:34599 (local address 127.0.0.1:41640)
0806 00:58:40.970774 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:40.970789 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:40.970908 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:40.971452 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:40.971455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:40.971659 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:40.971758 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:40.971763 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:40.972834 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:40.972837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:40.973495 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:40.973505 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:40.974195 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:40.974199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:40.974346 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:40.974841 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:40.974853 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:40.976536 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:40.978741 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:40.978745 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:40.978747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:40.978945 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:40.979284 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:40.979289 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:40.979291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:40.979374 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:40.980009 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:40.980012 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:40.980104 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 00:58:40.980221 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 00:58:40.980382 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":21}
I20260806 00:58:40.980666 25809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.970731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41640 (local address 127.27.77.253:34599)
0806 00:58:40.971113 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:40.971123 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:40.971140 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:58:40.971173 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:40.971177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:40.971332 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:40.971487 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:40.971878 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:40.972704 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:40.973608 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:40.974080 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:40.974198 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:40.976700 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:40.976718 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:40.976730 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:40.976752 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:40.978545 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:40.979101 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:40.979106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:40.979108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:40.979188 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:40.979484 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:40.979487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:40.979488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:40.979904 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:40.980051 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:40.980334 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 00:58:40.980502 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":277,"thread_start_us":175,"threads_started":1}
I20260806 00:58:40.980888 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.252:36691
--webserver_interface=127.27.77.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.252.keytab
--principal=kudu/127.27.77.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:35367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:37179,127.27.77.253:34599,127.27.77.252:36691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:58:40.983165 25803 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:58:40.983961 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.983688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:36691 (local address 127.0.0.1:39636)
0806 00:58:40.983810 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:40.983895 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 00:58:40.984169 25737 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
I20260806 00:58:40.985112 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:40.984899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:36691 (local address 127.0.0.1:39638)
0806 00:58:40.985006 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:40.985051 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 00:58:40.985278 25803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:36691: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
I20260806 00:58:41.034605 25803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } attempt: 1
I20260806 00:58:41.035779 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.035344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:36691 (local address 127.0.0.1:39652)
0806 00:58:41.035572 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:41.035669 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 00:58:41.036993 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.036718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:36691 (local address 127.0.0.1:39666)
0806 00:58:41.036853 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:41.036917 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 00:58:41.037204 25803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:36691: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
I20260806 00:58:41.079902 25704 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } attempt: 4
Aug 06 00:58:41 dist-test-slave-sb2z krb5kdc[25597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977920, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for kudu/127.27.77.253@KRBTEST.COM
I20260806 00:58:41.090101 25809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.080678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41650 (local address 127.27.77.253:34599)
0806 00:58:41.080819 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.080825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.080922 (+ 97us) server_negotiation.cc:407] Connection header received
0806 00:58:41.081125 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.081127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.081175 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.081265 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.081718 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.082621 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.083902 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.084158 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.084297 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.086711 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:41.086715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:41.086717 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:41.086741 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:41.088363 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.088811 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.088816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.088818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.088899 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.089199 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.089203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.089205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.089403 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:41.089515 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:41.089817 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 00:58:41.089934 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:58:41.090101 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.080569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:34599 (local address 127.0.0.1:41650)
0806 00:58:41.080802 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:41.080820 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:41.080985 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:41.081285 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:41.081288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:41.081495 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:41.081590 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:41.081595 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.082739 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.082744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:41.083772 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:41.083783 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.084311 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.084315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:41.084431 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.085011 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:41.085023 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:41.086560 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:41.088498 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:41.088502 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:41.088504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:41.088708 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:41.089013 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:41.089016 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:41.089018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:41.089107 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:41.089507 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:41.089511 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:41.089621 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 00:58:41.089760 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 00:58:41.089916 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260806 00:58:41.090975 25704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:58:41.091730 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.091464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:36691 (local address 127.0.0.1:39678)
0806 00:58:41.091607 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:41.091660 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 00:58:41.092581 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.092310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:36691 (local address 127.0.0.1:39690)
0806 00:58:41.092447 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:41.092524 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260806 00:58:41.092830 25704 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:36691: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
I20260806 00:58:41.105412 25803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } attempt: 2
W20260806 00:58:41.105993 25810 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
I20260806 00:58:41.106305 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.105972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:36691 (local address 127.0.0.1:39698)
0806 00:58:41.106141 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:41.106214 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 00:58:41.106400 25810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:41.106456 25810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:41.106536 25810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 00:58:41.107089 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.106848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:36691 (local address 127.0.0.1:39710)
0806 00:58:41.106966 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:41.107021 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 00:58:41.107228 25803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:36691: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
W20260806 00:58:41.110901 25810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:41.110966 25810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:41.110985 25810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:41.111003 25810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:41.111018 25810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:41.116549 25810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:35367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:37179,127.27.77.253:34599,127.27.77.252:36691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.252.keytab
--principal=kudu/127.27.77.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.252:36691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.77.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:41.117724 25810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:41.118757 25810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:41.127674 25819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:41.127674 25817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:41.127698 25816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:41.128069 25810 server_base.cc:1034] running on GCE node
I20260806 00:58:41.128599 25810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:41.129706 25810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:41.130877 25810 hybrid_clock.cc:648] HybridClock initialized: now 1785977921130858 us; error 32 us; skew 500 ppm
Aug 06 00:58:41 dist-test-slave-sb2z krb5kdc[25597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977921, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:41.133877 25810 init.cc:377] Logged in from keytab as kudu/127.27.77.252@KRBTEST.COM (short username kudu)
I20260806 00:58:41.134915 25810 webserver.cc:460] Webserver started at http://127.27.77.252:42083/ using document root <none> and password file <none>
I20260806 00:58:41.135516 25810 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:41.135684 25810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:41.135943 25810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:41.138173 25810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/data/instance:
uuid: "a54517277a9c4f45a100f1c428fe7426"
format_stamp: "Formatted at 2026-08-06 00:58:41 on dist-test-slave-sb2z"
server_key: "91d36423f2f7225a7e2cc0eac7531279"
server_key_iv: "19ba67b00348e2aeab806952e0ce17d6"
server_key_version: "encryptionkey@0"
I20260806 00:58:41.138795 25810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/wal/instance:
uuid: "a54517277a9c4f45a100f1c428fe7426"
format_stamp: "Formatted at 2026-08-06 00:58:41 on dist-test-slave-sb2z"
server_key: "91d36423f2f7225a7e2cc0eac7531279"
server_key_iv: "19ba67b00348e2aeab806952e0ce17d6"
server_key_version: "encryptionkey@0"
I20260806 00:58:41.142968 25810 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260806 00:58:41.145825 25829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:41.147275 25810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:58:41.147446 25810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/wal
uuid: "a54517277a9c4f45a100f1c428fe7426"
format_stamp: "Formatted at 2026-08-06 00:58:41 on dist-test-slave-sb2z"
server_key: "91d36423f2f7225a7e2cc0eac7531279"
server_key_iv: "19ba67b00348e2aeab806952e0ce17d6"
server_key_version: "encryptionkey@0"
I20260806 00:58:41.147655 25810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:41.152058 25704 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } attempt: 1
I20260806 00:58:41.152942 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.152615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:36691 (local address 127.0.0.1:46924)
0806 00:58:41.152766 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:41.152842 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 00:58:41.153782 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.153532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:36691 (local address 127.0.0.1:46926)
0806 00:58:41.153669 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:41.153726 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 00:58:41.153970 25704 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:36691: Network error: Client connection negotiation failed: client connection to 127.27.77.252:36691: connect: Connection refused (error 111)
I20260806 00:58:41.171279 25810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:41.174507 25810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:41.174731 25810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:41.182468 25810 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.252:36691
I20260806 00:58:41.182775 25904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.252:36691 every 8 connection(s)
I20260806 00:58:41.183658 25810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/data/info.pb
I20260806 00:58:41.186455 25905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:41.189625 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 25810
I20260806 00:58:41.189762 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977619051187-27959-0/minicluster-data/master-2/wal/instance
I20260806 00:58:41.190071 27959 external_mini_cluster.cc:1428] Setting key bbf94e09d8dd08705406eac0ed793853
I20260806 00:58:41.190754 25905 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:58:41.198505 25803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } attempt: 3
Aug 06 00:58:41 dist-test-slave-sb2z krb5kdc[25597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977921, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.252@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
Aug 06 00:58:41 dist-test-slave-sb2z krb5kdc[25597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.252@KRBTEST.COM
Aug 06 00:58:41 dist-test-slave-sb2z krb5kdc[25597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977920, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.253@KRBTEST.COM for kudu/127.27.77.252@KRBTEST.COM
I20260806 00:58:41.205588 25708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.192818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38048 (local address 127.27.77.254:37179)
0806 00:58:41.192971 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.193008 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.193718 (+ 710us) server_negotiation.cc:407] Connection header received
0806 00:58:41.193992 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.193996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.194053 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.194118 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.195353 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.195931 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.197381 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.197732 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.197850 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.200991 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:41.200998 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:41.201002 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:41.201040 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:41.202928 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.203622 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.203626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.203628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.203723 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.204121 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.204125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.204127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.204404 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:41.204511 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:41.205260 (+ 749us) server_negotiation.cc:299] Negotiation successful
0806 00:58:41.205409 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:58:41.207026 25909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.191644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46934 (local address 127.27.77.252:36691)
0806 00:58:41.192145 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.192160 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.192196 (+ 36us) server_negotiation.cc:407] Connection header received
0806 00:58:41.193201 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.193223 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.193645 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.193988 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.195127 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.196169 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.197381 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.198004 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.198085 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.201452 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:41.201458 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:41.201470 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:41.201493 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:41.203680 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.204247 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.204252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.204254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.204334 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.204684 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.204687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.204689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.205040 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:41.205319 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:41.205500 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 00:58:41.205894 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"spinlock_wait_cycles":15232,"thread_start_us":130,"threads_started":1}
I20260806 00:58:41.207032 25910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.192771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:37179 (local address 127.0.0.1:38048)
0806 00:58:41.193464 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:41.193564 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:41.193816 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:41.194360 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:41.194366 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:41.194849 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:41.195201 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:41.195215 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.196072 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.196077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:41.197242 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:41.197254 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.197898 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.197901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:41.198095 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.198729 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:41.198751 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:41.200756 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:41.203070 (+ 2314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:41.203079 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:41.203088 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:41.203458 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:41.203831 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:41.203836 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:41.203837 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:41.204029 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:41.204536 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:41.204543 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:41.204908 (+ 365us) client_negotiation.cc:769] Sending connection context
0806 00:58:41.205804 (+ 896us) client_negotiation.cc:240] Negotiation successful
0806 00:58:41.206027 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"spinlock_wait_cycles":9600,"thread_start_us":144,"threads_started":1}
I20260806 00:58:41.208652 25905 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:58:41.210649 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.199024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:36691 (local address 127.0.0.1:46948)
0806 00:58:41.199205 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:41.199222 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:41.199339 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:41.199897 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:41.199902 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:41.200102 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:41.200227 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:41.200236 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.201201 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.201206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:41.202062 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:41.202078 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.203644 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.203648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:41.203756 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.204320 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:41.204332 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:41.206170 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:41.208928 (+ 2758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:41.208931 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:41.208934 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:41.209153 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:41.209525 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:41.209529 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:41.209531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:41.209623 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:41.210089 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:41.210092 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:41.210198 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 00:58:41.210297 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 00:58:41.210471 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 00:58:41.211223 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.199205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46948 (local address 127.27.77.252:36691)
0806 00:58:41.199615 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.199627 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.199641 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:58:41.199679 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.199681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.199733 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.199841 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.200364 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.201092 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.203077 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.203457 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.203976 (+ 519us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.206314 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:41.206318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:41.206321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:41.206347 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:41.208798 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.209267 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.209270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.209272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.209397 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.209741 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.209744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.209745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.209968 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:41.210835 (+ 867us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:41.210910 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 00:58:41.211037 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":112,"threads_started":1}
Aug 06 00:58:41 dist-test-slave-sb2z krb5kdc[25597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977921, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.252@KRBTEST.COM for kudu/127.27.77.253@KRBTEST.COM
I20260806 00:58:41.215035 25803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df: Bootstrap starting.
Aug 06 00:58:41 dist-test-slave-sb2z krb5kdc[25597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:58:41.218246 25803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:41.219236 25809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.209394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52712 (local address 127.27.77.253:34599)
0806 00:58:41.209530 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.209535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.209547 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:58:41.209581 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.209584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.209624 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.209714 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.210259 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.211046 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.212003 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.212284 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.212387 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.214850 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:41.214856 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:41.214861 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:41.214898 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:41.216725 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.217650 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.217654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.217656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.217747 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.218056 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.218060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.218062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.218310 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:41.218602 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:41.218934 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 00:58:41.219076 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:58:41.219312 25910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.209143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:34599 (local address 127.0.0.1:52712)
0806 00:58:41.209391 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:41.209405 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:41.209480 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:41.209845 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:41.209848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:41.210033 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:41.210153 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:41.210160 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.211160 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.211165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:41.211873 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:41.211882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.212508 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.212514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:41.212640 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.213128 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:41.213142 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:41.214686 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:41.217202 (+ 2516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:41.217205 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:41.217208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:41.217484 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:41.217865 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:41.217868 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:41.217870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:41.217962 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:41.218587 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:41.218590 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:41.218719 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 00:58:41.218855 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 00:58:41.219090 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179}
I20260806 00:58:41.219194 25803 log.cc:824] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:41.220394 25905 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:58:41.221189 25704 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } attempt: 2
I20260806 00:58:41.221499 25708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.210371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38062 (local address 127.27.77.254:37179)
0806 00:58:41.210521 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.210526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.210572 (+ 46us) server_negotiation.cc:407] Connection header received
0806 00:58:41.210718 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.210721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.210771 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.210853 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.212159 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.212651 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.213594 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.213825 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.213947 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.216882 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:41.216887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:41.216890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:41.216920 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:41.219450 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.220066 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.220073 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.220075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.220179 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.220519 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.220523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.220524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.220769 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:41.221080 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:41.221170 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 00:58:41.221296 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:58:41.222877 25803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df: No bootstrap required, opened a new log
Aug 06 00:58:41 dist-test-slave-sb2z krb5kdc[25597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977921, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.252@KRBTEST.COM for kudu/127.27.77.252@KRBTEST.COM
I20260806 00:58:41.228256 25803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } }
I20260806 00:58:41.228570 25803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52e496d0f1df47909fadac5e3e7626df, State: Initialized, Role: FOLLOWER
Aug 06 00:58:41 dist-test-slave-sb2z krb5kdc[25597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977920, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for kudu/127.27.77.252@KRBTEST.COM
I20260806 00:58:41.228986 25803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } }
Aug 06 00:58:41 dist-test-slave-sb2z krb5kdc[25597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.253@KRBTEST.COM
I20260806 00:58:41.230794 25803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:41.233265 25910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.220979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:36691 (local address 127.0.0.1:46952)
0806 00:58:41.221099 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:41.221112 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:41.221505 (+ 393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:41.222037 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:41.222040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:41.222205 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:41.222296 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:41.222302 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.223313 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.223316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:41.224665 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:41.224677 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.225352 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.225355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:41.225445 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.225950 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:41.225964 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:41.227588 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:41.230585 (+ 2997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:41.230588 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:41.230591 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:41.230932 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:41.231517 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:41.231520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:41.231546 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:41.231643 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:41.232601 (+ 958us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:41.232605 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:41.232708 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 00:58:41.232814 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 00:58:41.232976 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 00:58:41.233295 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.221832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:36691 (local address 127.0.0.1:46968)
0806 00:58:41.221935 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:41.221951 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:41.222033 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:41.222607 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:41.222610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:41.222782 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:41.222914 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:41.222923 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.224442 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.224446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:41.225273 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:41.225283 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.226237 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.226241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:41.226329 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.226790 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:41.226804 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:41.229086 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:41.231436 (+ 2350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:41.231440 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:41.231443 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:41.231777 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:41.232137 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:41.232143 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:41.232145 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:41.232251 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:41.232752 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:41.232756 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:41.232868 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 00:58:41.232961 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 00:58:41.233110 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 00:58:41.234005 25909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.222260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46968 (local address 127.27.77.252:36691)
0806 00:58:41.222370 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.222374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.222389 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:58:41.222426 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.222431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.222478 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.223076 (+ 598us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.223176 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.224312 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.225416 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.226108 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.229255 (+ 3147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.229379 (+ 124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:41.229382 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:41.229385 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:41.229403 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:41.231298 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.231915 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.231919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.231920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.232006 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.232377 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.232380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.232383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.232624 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:41.233640 (+ 1016us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:41.233720 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 00:58:41.233838 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:58:41.234484 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.221182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46952 (local address 127.27.77.252:36691)
0806 00:58:41.221570 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.221576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.221591 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:58:41.221831 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.221833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.221877 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.221935 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.222410 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.223190 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.224799 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.225141 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.225243 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.227726 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:41.227729 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:41.227732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:41.227753 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:41.230154 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.231041 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.231045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.231047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.231390 (+ 343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.232003 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.232013 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.232016 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.232274 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:41.232389 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:41.233815 (+ 1426us) server_negotiation.cc:299] Negotiation successful
0806 00:58:41.233952 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299}
I20260806 00:58:41.235508 25809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.223247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52722 (local address 127.27.77.253:34599)
0806 00:58:41.223752 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.223756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.224014 (+ 258us) server_negotiation.cc:407] Connection header received
0806 00:58:41.224217 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.224220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.224261 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.224310 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.225187 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.225937 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.227061 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.227427 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.227641 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.230869 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:41.230872 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:41.230875 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:41.230905 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:41.232673 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.233185 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.233189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.233191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.233276 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.233672 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.233675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.233677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.233880 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:41.234068 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:41.235257 (+ 1189us) server_negotiation.cc:299] Negotiation successful
0806 00:58:41.235376 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368}
I20260806 00:58:41.231858 25915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } } }
I20260806 00:58:41.235919 25915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:58:41.237661 25704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531: Bootstrap starting.
I20260806 00:58:41.237661 25905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426: Bootstrap starting.
W20260806 00:58:41.238209 25928 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:58:41.238293 25928 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:58:41.240401 25704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:41.241998 25704 log.cc:824] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:41.242986 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:41.240425 25713 master_runner.cc:186] Error getting master registration for 127.27.77.254:37179: OK, instance_id { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" instance_seqno: 1785977920685936 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 00:58:41.244005 25713 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:58:41.244282 25704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531: No bootstrap required, opened a new log
I20260806 00:58:41.240401 25905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:41.245647 25905 log.cc:824] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:41.244390 25667 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "45e455f7040043c6ac5bc03441cde531"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38062
I20260806 00:58:41.247777 25704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } }
I20260806 00:58:41.248050 25704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e455f7040043c6ac5bc03441cde531, State: Initialized, Role: FOLLOWER
I20260806 00:58:41.248425 25704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } }
I20260806 00:58:41.248734 25905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426: No bootstrap required, opened a new log
I20260806 00:58:41.249119 25932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } } }
I20260806 00:58:41.249424 25932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:58:41.249903 25704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:41.250003 25667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 00:58:41.250119 25667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:41.251497 25667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } }
I20260806 00:58:41.252563 25667 leader_election.cc:288] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [CANDIDATE]: Term 1 election: Requested vote from peers 52e496d0f1df47909fadac5e3e7626df (127.27.77.253:34599), a54517277a9c4f45a100f1c428fe7426 (127.27.77.252:36691)
I20260806 00:58:41.253604 25766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "45e455f7040043c6ac5bc03441cde531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "52e496d0f1df47909fadac5e3e7626df"
I20260806 00:58:41.253880 25766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:41.255183 25905 raft_consensus.cc:348] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } }
I20260806 00:58:41.255409 25766 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e455f7040043c6ac5bc03441cde531 in term 1.
I20260806 00:58:41.255903 25637 leader_election.cc:302] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e455f7040043c6ac5bc03441cde531, 52e496d0f1df47909fadac5e3e7626df; no voters:
I20260806 00:58:41.255477 25905 raft_consensus.cc:729] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a54517277a9c4f45a100f1c428fe7426, State: Initialized, Role: FOLLOWER
I20260806 00:58:41.256436 25905 consensus_queue.cc:260] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } }
I20260806 00:58:41.257174 25932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:41.257567 25932 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [term 1 LEADER]: Becoming Leader. State: Replica: 45e455f7040043c6ac5bc03441cde531, State: Running, Role: LEADER
I20260806 00:58:41.258229 25905 sys_catalog.cc:564] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:41.258298 25872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "45e455f7040043c6ac5bc03441cde531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a54517277a9c4f45a100f1c428fe7426"
I20260806 00:58:41.258507 25872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:41.260759 25872 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e455f7040043c6ac5bc03441cde531 in term 1.
I20260806 00:58:41.261260 25952 sys_catalog.cc:455] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } } }
I20260806 00:58:41.262774 25952 sys_catalog.cc:458] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:58:41.264035 25610 master_runner.cc:263] No leader master found from master 45e455f7040043c6ac5bc03441cde531
I20260806 00:58:41.264113 25610 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:58:41.258107 25932 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } }
I20260806 00:58:41.266223 25932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45e455f7040043c6ac5bc03441cde531. Latest consensus state: current_term: 1 leader_uuid: "45e455f7040043c6ac5bc03441cde531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } } }
I20260806 00:58:41.266368 25932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [sys.catalog]: This master's current role is: LEADER
W20260806 00:58:41.269984 25967 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:58:41.270069 25967 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 00:58:41.270830 25945 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:58:41.270907 25945 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:58:41.271036 25809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.255349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52734 (local address 127.27.77.253:34599)
0806 00:58:41.255470 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.255475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.259871 (+ 4396us) server_negotiation.cc:407] Connection header received
0806 00:58:41.260086 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.260089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.260139 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.260190 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.261724 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.262503 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.264255 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.264618 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.264716 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.267008 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:41.267012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:41.267015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:41.267045 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:41.268915 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.269441 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.269446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.269451 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.269565 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.269913 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.269916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.269918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.270142 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:41.270258 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:41.270732 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 00:58:41.270861 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:58:41.275930 25971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:41.277041 25708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.256824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38074 (local address 127.27.77.254:37179)
0806 00:58:41.256932 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.256936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.261937 (+ 5001us) server_negotiation.cc:407] Connection header received
0806 00:58:41.261986 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.261989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.262041 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.262122 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.267404 (+ 5282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.268251 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.269395 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.269701 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.269778 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.272953 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:41.272958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:41.272961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:41.272989 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:41.274877 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.275398 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.275402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.275404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.275507 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.275941 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.275945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.275947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.276189 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:41.276324 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:41.276735 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 00:58:41.276875 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:58:41.277477 25971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:41.281822 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:41.256516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46970 (local address 127.27.77.252:36691)
0806 00:58:41.256653 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.256658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.263986 (+ 7328us) server_negotiation.cc:407] Connection header received
0806 00:58:41.264111 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.264117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.264171 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.264248 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.266353 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.267111 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.275675 (+ 8564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:41.276046 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:41.276158 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:41.277881 (+ 1723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:41.277885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:41.277888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:41.277919 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:41.279717 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.280287 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.280292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.280294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.280396 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:41.280752 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:41.280756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:41.280758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:41.281026 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:41.281188 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:41.281487 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 00:58:41.281639 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:58:41.289129 25872 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426 [term 1 FOLLOWER]: Refusing update from remote peer 45e455f7040043c6ac5bc03441cde531: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:58:41.289420 25766 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df [term 1 FOLLOWER]: Refusing update from remote peer 45e455f7040043c6ac5bc03441cde531: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:58:41.289924 25956 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:58:41.294620 25952 sys_catalog.cc:455] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45e455f7040043c6ac5bc03441cde531. Latest consensus state: current_term: 1 leader_uuid: "45e455f7040043c6ac5bc03441cde531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } } }
I20260806 00:58:41.294780 25952 sys_catalog.cc:458] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:58:41.296435 25956 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:58:41.301654 25915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45e455f7040043c6ac5bc03441cde531. Latest consensus state: current_term: 1 leader_uuid: "45e455f7040043c6ac5bc03441cde531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } } }
I20260806 00:58:41.301823 25915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:58:41.305131 25956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45e455f7040043c6ac5bc03441cde531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } } }
I20260806 00:58:41.305274 25956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:41.305402 25973 mvcc.cc:205] Tried to move back new op lower bound from 7315365565592518656 to 7315365565502169088. Current Snapshot: MvccSnapshot[applied={T|T < 7315365565592518656}]
I20260806 00:58:41.307088 25952 sys_catalog.cc:455] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45e455f7040043c6ac5bc03441cde531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } } }
I20260806 00:58:41.307214 25952 sys_catalog.cc:458] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:58:41.308038 25975 mvcc.cc:205] Tried to move back new op lower bound from 7315365565592518656 to 7315365565502169088. Current Snapshot: MvccSnapshot[applied={T|T < 7315365565592518656}]
I20260806 00:58:41.309358 25932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45e455f7040043c6ac5bc03441cde531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } } }
I20260806 00:58:41.309485 25932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531 [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:41.310351 25915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45e455f7040043c6ac5bc03441cde531" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e455f7040043c6ac5bc03441cde531" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 37179 } } peers { permanent_uuid: "52e496d0f1df47909fadac5e3e7626df" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 34599 } } peers { permanent_uuid: "a54517277a9c4f45a100f1c428fe7426" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 36691 } } }
I20260806 00:58:41.310483 25915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:58:41.326148 25971 catalog_manager.cc:1347] Generated new cluster ID: 72fd0b53607f405384198af4ebf29788
I20260806 00:58:41.326249 25971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:41.346004 25978 mvcc.cc:205] Tried to move back new op lower bound from 7315365565592518656 to 7315365565502169088. Current Snapshot: MvccSnapshot[applied={T|T < 7315365565592518656}]
I20260806 00:58:41.384631 25971 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:41.385964 25971 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:41.401829 25971 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45e455f7040043c6ac5bc03441cde531: Generated new TSK 0
I20260806 00:58:41.402712 25971 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:58:41.441440 25809 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:41.430358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52742 (local address 127.27.77.253:34599)
0806 00:58:41.430541 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.430547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.439993 (+ 9446us) server_negotiation.cc:407] Connection header received
0806 00:58:41.440467 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.440470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.440525 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.440624 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.441283 (+ 659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52742: BlockingRecv error: recv got EOF from 127.0.0.1:52742 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260806 00:58:41.441440 25708 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:41.431499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38080 (local address 127.27.77.254:37179)
0806 00:58:41.431654 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.431658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.440319 (+ 8661us) server_negotiation.cc:407] Connection header received
0806 00:58:41.440517 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.440521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.440594 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.440654 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.441283 (+ 629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38080: BlockingRecv error: recv got EOF from 127.0.0.1:38080 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260806 00:58:41.442409 25911 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:41.431117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46980 (local address 127.27.77.252:36691)
0806 00:58:41.431229 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.431234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.440058 (+ 8824us) server_negotiation.cc:407] Connection header received
0806 00:58:41.440133 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.440137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.440205 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.440317 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.442286 (+ 1969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46980: BlockingRecv error: recv got EOF from 127.0.0.1:46980 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260806 00:58:41.447512 25809 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:41.445779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52748 (local address 127.27.77.253:34599)
0806 00:58:41.445936 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.445942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.446294 (+ 352us) server_negotiation.cc:407] Connection header received
0806 00:58:41.446680 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.446684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.446767 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.446825 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.447394 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52748: BlockingRecv error: recv got EOF from 127.0.0.1:52748 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 00:58:41.453459 25809 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:41.451743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52758 (local address 127.27.77.253:34599)
0806 00:58:41.451891 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.451895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.452437 (+ 542us) server_negotiation.cc:407] Connection header received
0806 00:58:41.452647 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.452651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.452705 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.452760 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.453330 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52758: BlockingRecv error: recv got EOF from 127.0.0.1:52758 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 00:58:41.459684 25809 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:41.457915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52766 (local address 127.27.77.253:34599)
0806 00:58:41.458020 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.458024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.458585 (+ 561us) server_negotiation.cc:407] Connection header received
0806 00:58:41.458841 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.458845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.458916 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.458980 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.459521 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52766: BlockingRecv error: recv got EOF from 127.0.0.1:52766 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 00:58:41.469765 25809 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:41.468054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52768 (local address 127.27.77.253:34599)
0806 00:58:41.468207 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.468212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.468751 (+ 539us) server_negotiation.cc:407] Connection header received
0806 00:58:41.468947 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.468951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.469007 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.469060 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.469640 (+ 580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52768: BlockingRecv error: recv got EOF from 127.0.0.1:52768 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260806 00:58:41.481856 25809 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:41.480497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52770 (local address 127.27.77.253:34599)
0806 00:58:41.480632 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.480637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.480927 (+ 290us) server_negotiation.cc:407] Connection header received
0806 00:58:41.481097 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.481107 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.481157 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.481250 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.481734 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52770: BlockingRecv error: recv got EOF from 127.0.0.1:52770 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 00:58:41.501322 25809 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:41.500006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52786 (local address 127.27.77.253:34599)
0806 00:58:41.500175 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.500180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.500366 (+ 186us) server_negotiation.cc:407] Connection header received
0806 00:58:41.500521 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.500523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.500573 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.500637 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.501194 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52786: BlockingRecv error: recv got EOF from 127.0.0.1:52786 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 00:58:41.537307 25809 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:41.535899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52794 (local address 127.27.77.253:34599)
0806 00:58:41.536095 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.536101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.536281 (+ 180us) server_negotiation.cc:407] Connection header received
0806 00:58:41.536497 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.536500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.536552 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.536652 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.537150 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52794: BlockingRecv error: recv got EOF from 127.0.0.1:52794 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 00:58:41.605053 25809 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:41.603778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52796 (local address 127.27.77.253:34599)
0806 00:58:41.603948 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.603952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.604091 (+ 139us) server_negotiation.cc:407] Connection header received
0806 00:58:41.604276 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.604279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.604337 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.604434 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.604912 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52796: BlockingRecv error: recv got EOF from 127.0.0.1:52796 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 00:58:41.737329 25809 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:41.736014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52802 (local address 127.27.77.253:34599)
0806 00:58:41.736204 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.736209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.736357 (+ 148us) server_negotiation.cc:407] Connection header received
0806 00:58:41.736547 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.736551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.736601 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.736692 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.737189 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52802: BlockingRecv error: recv got EOF from 127.0.0.1:52802 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260806 00:58:41.997169 25809 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:41.996001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52804 (local address 127.27.77.253:34599)
0806 00:58:41.996177 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:41.996181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:41.996216 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:58:41.996368 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:41.996370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:41.996420 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:41.996507 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:41.996975 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52804: BlockingRecv error: recv got EOF from 127.0.0.1:52804 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 00:58:42.239841 25928 catalog_manager.cc:1259] Loaded cluster ID: 72fd0b53607f405384198af4ebf29788
I20260806 00:58:42.239943 25928 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df: loading cluster ID for follower catalog manager: success
I20260806 00:58:42.241734 25928 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df: acquiring CA information for follower catalog manager: success
I20260806 00:58:42.243018 25928 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 52e496d0f1df47909fadac5e3e7626df: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:58:42.271550 25967 catalog_manager.cc:1259] Loaded cluster ID: 72fd0b53607f405384198af4ebf29788
I20260806 00:58:42.271687 25967 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426: loading cluster ID for follower catalog manager: success
I20260806 00:58:42.273496 25967 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426: acquiring CA information for follower catalog manager: success
I20260806 00:58:42.274716 25967 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a54517277a9c4f45a100f1c428fe7426: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:58:42.516007 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:42.511680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52814 (local address 127.27.77.253:34599)
0806 00:58:42.512009 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:42.512014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:42.512104 (+ 90us) server_negotiation.cc:407] Connection header received
0806 00:58:42.512230 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:42.512232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:42.512388 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:42.512512 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:58:42.512778 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:42.513407 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:42.514341 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:42.514617 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:42.514711 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:42.515031 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:58:42.515517 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:58:42.515653 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:42.515762 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 00:58:42.515820 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":109,"threads_started":1}
I20260806 00:58:42.523353 26084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:42.519614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46994 (local address 127.27.77.252:36691)
0806 00:58:42.519866 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:42.519870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:42.519883 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:58:42.519968 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:42.519970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:42.520110 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:42.520225 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:58:42.520456 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:42.521009 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:42.521882 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:42.522128 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:42.522213 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:42.522491 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:58:42.522944 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:58:42.523044 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:42.523174 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 00:58:42.523228 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":105,"threads_started":1}
I20260806 00:58:42.531384 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:42.526760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52820 (local address 127.27.77.253:34599)
0806 00:58:42.526893 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:42.526898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:42.527218 (+ 320us) server_negotiation.cc:407] Connection header received
0806 00:58:42.527482 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:42.527486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:42.527574 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:42.527656 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:58:42.527976 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:42.528701 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:42.529860 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:42.530207 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:42.530333 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:42.530712 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:58:42.530801 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:58:42.530849 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:42.531156 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 00:58:42.531231 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:58:42.531875 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:42.526735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38096 (local address 127.27.77.254:37179)
0806 00:58:42.526970 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:42.526974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:42.527203 (+ 229us) server_negotiation.cc:407] Connection header received
0806 00:58:42.527434 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:42.527439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:42.527641 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:42.527776 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:58:42.528290 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:42.529135 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:42.530165 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:42.530512 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:42.530632 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:42.530966 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:58:42.531327 (+ 361us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:58:42.531419 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:42.531628 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 00:58:42.531705 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":73,"threads_started":1}
I20260806 00:58:42.531908 26084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:42.527422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47000 (local address 127.27.77.252:36691)
0806 00:58:42.527559 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:42.527564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:42.527954 (+ 390us) server_negotiation.cc:407] Connection header received
0806 00:58:42.528134 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:42.528140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:42.528188 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:42.528271 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:58:42.528493 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:42.529230 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:42.530490 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:42.530794 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:42.530875 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:42.531295 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:58:42.531367 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:58:42.531413 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:42.531678 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 00:58:42.531734 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 00:58:42.536777 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 25610
I20260806 00:58:42.553248 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 25713
I20260806 00:58:42.562990 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 25810
2026-08-06T00:58:42Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3435 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:58:42 dist-test-slave-sb2z krb5kdc[26097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:58:42 dist-test-slave-sb2z krb5kdc[26097](info): set up 2 sockets
Aug 06 00:58:42 dist-test-slave-sb2z krb5kdc[26097](info): commencing operation
krb5kdc: starting...
W20260806 00:58:43.810707 27959 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/test-user.keytab.
Aug 06 00:58:43 dist-test-slave-sb2z krb5kdc[26097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:58:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:58:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.77.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
Entry for principal kudu/127.27.77.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab.
I20260806 00:58:43.954731 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:46449
--webserver_interface=127.27.77.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:46449,127.27.77.253:46329,127.27.77.252:42393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:58:44.074653 26116 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:58:44.075042 26116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:44.075124 26116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:44.075202 26116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:44.079236 26116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:44.079339 26116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:44.079384 26116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:44.079423 26116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:44.079459 26116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:44.084893 26116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:46449,127.27.77.253:46329,127.27.77.252:42393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.254.keytab
--principal=kudu/127.27.77.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.254:46449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.77.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:44.086091 26116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:44.087186 26116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:44.096462 26125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:44.096488 26123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:44.096529 26121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:44.096913 26116 server_base.cc:1034] running on GCE node
I20260806 00:58:44.097520 26116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:44.098637 26116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:44.099805 26116 hybrid_clock.cc:648] HybridClock initialized: now 1785977924099785 us; error 36 us; skew 500 ppm
Aug 06 00:58:44 dist-test-slave-sb2z krb5kdc[26097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977924, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:44.102783 26116 init.cc:377] Logged in from keytab as kudu/127.27.77.254@KRBTEST.COM (short username kudu)
I20260806 00:58:44.103783 26116 webserver.cc:460] Webserver started at http://127.27.77.254:45685/ using document root <none> and password file <none>
I20260806 00:58:44.104365 26116 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:44.104461 26116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:44.104710 26116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:44.106904 26116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/data/instance:
uuid: "4607888aa08f430ab0ffec961689edaa"
format_stamp: "Formatted at 2026-08-06 00:58:44 on dist-test-slave-sb2z"
server_key: "bd8332ab4a9db6c3a0a5dbad12a074a2"
server_key_iv: "00293ada73fdf41c90b389d9a7d56acc"
server_key_version: "encryptionkey@0"
I20260806 00:58:44.107511 26116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/wal/instance:
uuid: "4607888aa08f430ab0ffec961689edaa"
format_stamp: "Formatted at 2026-08-06 00:58:44 on dist-test-slave-sb2z"
server_key: "bd8332ab4a9db6c3a0a5dbad12a074a2"
server_key_iv: "00293ada73fdf41c90b389d9a7d56acc"
server_key_version: "encryptionkey@0"
I20260806 00:58:44.111367 26116 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260806 00:58:44.114094 26134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:44.115644 26116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:58:44.115804 26116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/wal
uuid: "4607888aa08f430ab0ffec961689edaa"
format_stamp: "Formatted at 2026-08-06 00:58:44 on dist-test-slave-sb2z"
server_key: "bd8332ab4a9db6c3a0a5dbad12a074a2"
server_key_iv: "00293ada73fdf41c90b389d9a7d56acc"
server_key_version: "encryptionkey@0"
I20260806 00:58:44.115932 26116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:44.125499 26116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:44.128574 26116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:44.128791 26116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:44.136373 26116 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.254:46449
I20260806 00:58:44.136684 26204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.254:46449 every 8 connection(s)
I20260806 00:58:44.137486 26116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/data/info.pb
I20260806 00:58:44.140529 26205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:44.143409 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 26116
I20260806 00:58:44.143591 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-0/wal/instance
I20260806 00:58:44.143900 27959 external_mini_cluster.cc:1428] Setting key 97a9188160b79ce98a8ff187388a5e88
I20260806 00:58:44.145028 26205 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.77.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.253@KRBTEST.COM" created.
Aug 06 00:58:44 dist-test-slave-sb2z krb5kdc[26097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977924, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:58:44.165349 26210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.148045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43602 (local address 127.27.77.254:46449)
0806 00:58:44.148706 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.148722 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.148776 (+ 54us) server_negotiation.cc:407] Connection header received
0806 00:58:44.150077 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.150086 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.150405 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.150653 (+ 248us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.151596 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.153225 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.154298 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.154980 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.155202 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.159605 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:44.159613 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:44.159636 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:44.159663 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:44.161624 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.162196 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.162206 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.162212 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.162380 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.162764 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.162771 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.162776 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.163419 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:44.163625 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:44.164137 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 00:58:44.164469 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":495,"thread_start_us":177,"threads_started":1}
I20260806 00:58:44.165354 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.148044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:46449 (local address 127.0.0.1:43602)
0806 00:58:44.148608 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.148666 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:44.149669 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:44.150652 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:44.150657 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:44.151043 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:44.151351 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.151365 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.153386 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.153389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.154153 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.154163 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.155205 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.155208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.155389 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.156103 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:44.156120 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:44.159385 (+ 3265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:44.161773 (+ 2388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.161783 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.161796 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.162060 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.162506 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.162509 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.162511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.162643 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.163630 (+ 987us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:44.163636 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:44.163818 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 00:58:44.163973 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 00:58:44.164337 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":65,"thread_start_us":161,"threads_started":1}
I20260806 00:58:44.168252 26205 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:58:44.169129 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.168815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:46329 (local address 127.0.0.1:53142)
0806 00:58:44.168948 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.169063 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:46329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 00:58:44.169368 26135 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.77.253:46329: connect: Connection refused (error 111)
I20260806 00:58:44.170220 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.170009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:46329 (local address 127.0.0.1:53148)
0806 00:58:44.170120 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.170171 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:46329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 00:58:44.170496 26205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.253:46329: Network error: Client connection negotiation failed: client connection to 127.27.77.253:46329: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.253.keytab.
Entry for principal kudu/127.27.77.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.253.keytab.
I20260806 00:58:44.179610 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.253:46329
--webserver_interface=127.27.77.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.253.keytab
--principal=kudu/127.27.77.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:46449,127.27.77.253:46329,127.27.77.252:42393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:58:44.219805 26205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } attempt: 1
I20260806 00:58:44.220836 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.220525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:46329 (local address 127.0.0.1:53150)
0806 00:58:44.220691 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.220756 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:46329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 00:58:44.221643 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.221455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:46329 (local address 127.0.0.1:53164)
0806 00:58:44.221553 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.221596 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:46329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 00:58:44.221849 26205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.253:46329: Network error: Client connection negotiation failed: client connection to 127.27.77.253:46329: connect: Connection refused (error 111)
I20260806 00:58:44.290074 26205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } attempt: 2
I20260806 00:58:44.291042 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.290717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:46329 (local address 127.0.0.1:53172)
0806 00:58:44.290881 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.290955 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:46329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 00:58:44.292063 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.291732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:46329 (local address 127.0.0.1:53182)
0806 00:58:44.291874 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.291935 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.253:46329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 00:58:44.292260 26205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.253:46329: Network error: Client connection negotiation failed: client connection to 127.27.77.253:46329: connect: Connection refused (error 111)
W20260806 00:58:44.298800 26215 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:58:44.299149 26215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:44.299214 26215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:44.299293 26215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:44.303462 26215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:44.303573 26215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:44.303628 26215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:44.303661 26215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:44.303709 26215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:44.309199 26215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:46449,127.27.77.253:46329,127.27.77.252:42393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.253.keytab
--principal=kudu/127.27.77.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.253:46329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.77.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:44.310313 26215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:44.311336 26215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:58:44.320765 26221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:44.320776 26225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:44.320776 26223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:44.321019 26215 server_base.cc:1034] running on GCE node
I20260806 00:58:44.321774 26215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:44.322919 26215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:44.324127 26215 hybrid_clock.cc:648] HybridClock initialized: now 1785977924324093 us; error 46 us; skew 500 ppm
Aug 06 00:58:44 dist-test-slave-sb2z krb5kdc[26097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977924, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:44.327251 26215 init.cc:377] Logged in from keytab as kudu/127.27.77.253@KRBTEST.COM (short username kudu)
I20260806 00:58:44.328251 26215 webserver.cc:460] Webserver started at http://127.27.77.253:42169/ using document root <none> and password file <none>
I20260806 00:58:44.328795 26215 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:44.328855 26215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:44.329022 26215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:44.331072 26215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/data/instance:
uuid: "574ea8d990cf4fcbb6ea04737d3a56b2"
format_stamp: "Formatted at 2026-08-06 00:58:44 on dist-test-slave-sb2z"
server_key: "1373b0be18c5c6d74984b54455870323"
server_key_iv: "40150040a49baac17258a53f4d064d75"
server_key_version: "encryptionkey@0"
I20260806 00:58:44.331686 26215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/wal/instance:
uuid: "574ea8d990cf4fcbb6ea04737d3a56b2"
format_stamp: "Formatted at 2026-08-06 00:58:44 on dist-test-slave-sb2z"
server_key: "1373b0be18c5c6d74984b54455870323"
server_key_iv: "40150040a49baac17258a53f4d064d75"
server_key_version: "encryptionkey@0"
I20260806 00:58:44.335515 26215 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 00:58:44.338272 26234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:44.339823 26215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 00:58:44.339986 26215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/wal
uuid: "574ea8d990cf4fcbb6ea04737d3a56b2"
format_stamp: "Formatted at 2026-08-06 00:58:44 on dist-test-slave-sb2z"
server_key: "1373b0be18c5c6d74984b54455870323"
server_key_iv: "40150040a49baac17258a53f4d064d75"
server_key_version: "encryptionkey@0"
I20260806 00:58:44.340114 26215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:44.359187 26215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:44.363071 26215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:44.363325 26215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:44.371407 26215 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.253:46329
I20260806 00:58:44.371797 26306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.253:46329 every 8 connection(s)
I20260806 00:58:44.372509 26215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/data/info.pb
I20260806 00:58:44.375608 26307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:44.378016 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 26215
I20260806 00:58:44.378198 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-1/wal/instance
I20260806 00:58:44.378511 27959 external_mini_cluster.cc:1428] Setting key 39599a9432efecfd63ae9f6e7fad2909
I20260806 00:58:44.380081 26307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:58:44.383435 26205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } attempt: 3
WARNING: no policy specified for kudu/127.27.77.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.77.252@KRBTEST.COM" created.
Aug 06 00:58:44 dist-test-slave-sb2z krb5kdc[26097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977924, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.253@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
Aug 06 00:58:44 dist-test-slave-sb2z krb5kdc[26097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977924, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for kudu/127.27.77.253@KRBTEST.COM
I20260806 00:58:44.397617 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.383962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:46329 (local address 127.0.0.1:53192)
0806 00:58:44.384169 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.384186 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:44.384280 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:44.385421 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:44.385425 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:44.385645 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:44.385789 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.385798 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.387664 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.387668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.388537 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.388549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.389231 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.389235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.389357 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.389940 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:44.389953 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:44.393373 (+ 3420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:44.395474 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.395478 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.395480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.395823 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.396188 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.396192 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.396194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.396282 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.397053 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:44.397056 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:44.397167 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 00:58:44.397279 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 00:58:44.397435 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260806 00:58:44.398676 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.384066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53192 (local address 127.27.77.253:46329)
0806 00:58:44.384586 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.384601 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.384650 (+ 49us) server_negotiation.cc:407] Connection header received
0806 00:58:44.384829 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.384854 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.385147 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.385427 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.385925 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.387495 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.388674 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.389088 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.389227 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.393548 (+ 4321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:44.393561 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:44.393578 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:44.393608 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:44.395331 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.395943 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.395951 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.395963 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.396083 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.396379 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.396383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.396384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.396933 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:44.397091 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:44.397440 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 00:58:44.397744 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":365,"thread_start_us":164,"threads_started":1}
I20260806 00:58:44.398679 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.382668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:46449 (local address 127.0.0.1:43614)
0806 00:58:44.383211 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.383271 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:44.384748 (+ 1477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:44.385230 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:44.385240 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:44.385796 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:44.386196 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.386210 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.387043 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.387048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.388177 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.388190 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.388958 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.388962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.389126 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.389805 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:44.389830 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:44.392883 (+ 3053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:44.396237 (+ 3354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.396250 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.396260 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.396598 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.396924 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.396930 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.396934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.397048 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.397566 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:44.397576 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:44.397719 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 00:58:44.397861 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 00:58:44.398118 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":28,"thread_start_us":163,"threads_started":1}
I20260806 00:58:44.399859 26210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.382214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43614 (local address 127.27.77.254:46449)
0806 00:58:44.382398 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.382403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.383431 (+ 1028us) server_negotiation.cc:407] Connection header received
0806 00:58:44.384985 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.384988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.385060 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.385176 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.386350 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.386913 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.388333 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.388766 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.388898 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.393077 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:44.393083 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:44.393089 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:44.393118 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:44.396108 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.396729 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.396733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.396734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.396815 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.397172 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.397178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.397183 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.397435 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:44.399309 (+ 1874us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:44.399432 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 00:58:44.399628 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 00:58:44.401126 26307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:58:44.401480 26205 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:58:44.402187 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.401921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58744)
0806 00:58:44.402091 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.402134 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260806 00:58:44.402971 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.402721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58756)
0806 00:58:44.402877 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.402924 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260806 00:58:44.403147 26205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:42393: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Aug 06 00:58:44 dist-test-slave-sb2z krb5kdc[26097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977924, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.253@KRBTEST.COM for kudu/127.27.77.253@KRBTEST.COM
I20260806 00:58:44.410876 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.401686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:46329 (local address 127.0.0.1:53198)
0806 00:58:44.401846 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.401864 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:44.401937 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:44.402629 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:44.402634 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:44.402836 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:44.402971 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.402982 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.403994 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.403997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.404678 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.404688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.405254 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.405258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.405353 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.405855 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:44.405876 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:44.407359 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:44.409261 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.409264 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.409265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.409486 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.409812 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.409815 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.409816 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.409895 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.410322 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:44.410324 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:44.410408 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 00:58:44.410494 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 00:58:44.410718 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 00:58:44.410913 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.401910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53198 (local address 127.27.77.253:46329)
0806 00:58:44.402381 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.402384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.402416 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:58:44.402452 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.402455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.402496 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.402579 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.403124 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.403885 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.404819 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.405107 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.405195 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.407526 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:44.407532 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:44.407564 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:44.407596 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:44.409140 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.409644 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.409647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.409648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.409724 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.410001 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.410033 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.410035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.410230 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:44.410308 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:44.410658 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 00:58:44.410775 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396}
I20260806 00:58:44.411836 26307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:58:44.412801 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.412430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58764)
0806 00:58:44.412602 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.412726 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 00:58:44.413122 26238 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.77.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.252.keytab.
Entry for principal kudu/127.27.77.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.252.keytab.
I20260806 00:58:44.414161 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.413934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58766)
0806 00:58:44.414042 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.414097 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 00:58:44.414295 26307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:42393: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
I20260806 00:58:44.415023 27959 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
/tmp/dist-test-taskxB88Ks/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.252:42393
--webserver_interface=127.27.77.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.252.keytab
--principal=kudu/127.27.77.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.77.212:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.77.254:46449,127.27.77.253:46329,127.27.77.252:42393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:58:44.434341 26205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } attempt: 1
I20260806 00:58:44.435382 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.435019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58782)
0806 00:58:44.435225 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.435293 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 00:58:44.436257 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.436027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58792)
0806 00:58:44.436149 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.436204 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 00:58:44.436403 26205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:42393: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
I20260806 00:58:44.463629 26307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } attempt: 1
I20260806 00:58:44.464704 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.464305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58808)
0806 00:58:44.464522 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.464595 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 00:58:44.465517 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.465280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58818)
0806 00:58:44.465400 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.465456 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 00:58:44.465657 26307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:42393: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
I20260806 00:58:44.511621 26205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } attempt: 2
I20260806 00:58:44.512625 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.512286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58830)
0806 00:58:44.512445 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.512527 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 00:58:44.513398 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.513182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58844)
0806 00:58:44.513296 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.513348 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 00:58:44.513527 26205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:42393: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
I20260806 00:58:44.533838 26307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } attempt: 2
I20260806 00:58:44.534802 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.534433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58854)
0806 00:58:44.534618 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.534699 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 00:58:44.535712 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.535448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58858)
0806 00:58:44.535583 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.535639 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 00:58:44.535898 26307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:42393: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
W20260806 00:58:44.536258 26317 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:58:44.536561 26317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:58:44.536587 26317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:58:44.536629 26317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:58:44.540952 26317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:58:44.541019 26317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:58:44.541039 26317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:58:44.541057 26317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:58:44.541101 26317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:58:44.546880 26317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.77.212:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.77.254:46449,127.27.77.253:46329,127.27.77.252:42393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/krb5kdc/kudu_127.27.77.252.keytab
--principal=kudu/127.27.77.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.77.252:42393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.77.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:58:44.548592 26317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:58:44.549908 26317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:58:44.559427 26317 server_base.cc:1034] running on GCE node
W20260806 00:58:44.559343 26325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:44.559343 26322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:58:44.559337 26323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:58:44.560254 26317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:58:44.561477 26317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:58:44.562698 26317 hybrid_clock.cc:648] HybridClock initialized: now 1785977924562682 us; error 28 us; skew 500 ppm
Aug 06 00:58:44 dist-test-slave-sb2z krb5kdc[26097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977924, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:58:44.565886 26317 init.cc:377] Logged in from keytab as kudu/127.27.77.252@KRBTEST.COM (short username kudu)
I20260806 00:58:44.566897 26317 webserver.cc:460] Webserver started at http://127.27.77.252:45691/ using document root <none> and password file <none>
I20260806 00:58:44.567500 26317 fs_manager.cc:359] Metadata directory not provided
I20260806 00:58:44.567657 26317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:58:44.567853 26317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:58:44.569929 26317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/data/instance:
uuid: "5b32a902e36b433f844bd36a97035587"
format_stamp: "Formatted at 2026-08-06 00:58:44 on dist-test-slave-sb2z"
server_key: "f47e8b488deb1ae778b2d1d9d7af57bf"
server_key_iv: "eebf5004b807ae00dbcf13191bc44c2b"
server_key_version: "encryptionkey@0"
I20260806 00:58:44.570461 26317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/wal/instance:
uuid: "5b32a902e36b433f844bd36a97035587"
format_stamp: "Formatted at 2026-08-06 00:58:44 on dist-test-slave-sb2z"
server_key: "f47e8b488deb1ae778b2d1d9d7af57bf"
server_key_iv: "eebf5004b807ae00dbcf13191bc44c2b"
server_key_version: "encryptionkey@0"
I20260806 00:58:44.574422 26317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:58:44.577178 26333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:58:44.578526 26317 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:58:44.578651 26317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/data,/tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/wal
uuid: "5b32a902e36b433f844bd36a97035587"
format_stamp: "Formatted at 2026-08-06 00:58:44 on dist-test-slave-sb2z"
server_key: "f47e8b488deb1ae778b2d1d9d7af57bf"
server_key_iv: "eebf5004b807ae00dbcf13191bc44c2b"
server_key_version: "encryptionkey@0"
I20260806 00:58:44.578748 26317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:58:44.612726 26205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } attempt: 3
I20260806 00:58:44.613761 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.613413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58866)
0806 00:58:44.613585 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.613674 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 00:58:44.614593 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.614372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58876)
0806 00:58:44.614485 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.614539 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 00:58:44.614754 26205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:42393: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
I20260806 00:58:44.619622 26317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:58:44.623427 26317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:58:44.623688 26317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:58:44.627138 26307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } attempt: 3
I20260806 00:58:44.628034 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.627732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58878)
0806 00:58:44.627882 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.627946 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 00:58:44.629117 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.628798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58894)
0806 00:58:44.628983 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.629039 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260806 00:58:44.629304 26307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.77.252:42393: Network error: Client connection negotiation failed: client connection to 127.27.77.252:42393: connect: Connection refused (error 111)
I20260806 00:58:44.632092 26317 rpc_server.cc:307] RPC server started. Bound to: 127.27.77.252:42393
I20260806 00:58:44.632310 26404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.77.252:42393 every 8 connection(s)
I20260806 00:58:44.633232 26317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/data/info.pb
I20260806 00:58:44.633644 27959 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu as pid 26317
I20260806 00:58:44.633808 27959 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxB88Ks/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977619051187-27959-0/minicluster-data/master-2/wal/instance
I20260806 00:58:44.634212 27959 external_mini_cluster.cc:1428] Setting key de54a162a7c130cd5298fbf3fd857d95
I20260806 00:58:44.637036 26405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:58:44.642347 26405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:58:44 dist-test-slave-sb2z krb5kdc[26097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.252@KRBTEST.COM
Aug 06 00:58:44 dist-test-slave-sb2z krb5kdc[26097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977924, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.252@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
I20260806 00:58:44.651041 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.635796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58898 (local address 127.27.77.252:42393)
0806 00:58:44.636218 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.636237 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.636280 (+ 43us) server_negotiation.cc:407] Connection header received
0806 00:58:44.637171 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.637195 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.637634 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.638120 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.638781 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.640327 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.641594 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.642264 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.642445 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.645547 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:44.645553 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:44.645562 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:44.645597 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:44.647417 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.647921 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.647931 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.647936 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.648046 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.648396 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.648401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.648405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.648989 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:44.649206 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:44.649587 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 00:58:44.650100 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":112,"threads_started":1}
I20260806 00:58:44.653988 26210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.644445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43616 (local address 127.27.77.254:46449)
0806 00:58:44.644602 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.644609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.645023 (+ 414us) server_negotiation.cc:407] Connection header received
0806 00:58:44.645196 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.645202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.645267 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.645360 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.646099 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.646602 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.647477 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.647778 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.647899 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.650486 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:44.650489 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:44.650491 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:44.650519 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:44.652084 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.652572 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.652576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.652578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.652668 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.652948 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.652951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.652953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.653244 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:44.653368 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:44.653706 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 00:58:44.653823 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:58:44.654027 26411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.644382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.254:46449 (local address 127.0.0.1:43616)
0806 00:58:44.644885 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.644945 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:44.645080 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:44.645414 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:44.645420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:44.645822 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:44.645958 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.645965 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.646725 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.646729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.647346 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.647355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.647899 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.647903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.648025 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.648584 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:44.648599 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:44.650316 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:44.652196 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.652208 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.652217 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.652464 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.652766 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.652768 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.652770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.652851 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.653366 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:44.653373 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:44.653494 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 00:58:44.653645 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 00:58:44.653825 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"spinlock_wait_cycles":1920,"thread_start_us":219,"threads_started":1}
I20260806 00:58:44.655258 26405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:58:44 dist-test-slave-sb2z krb5kdc[26097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.254@KRBTEST.COM
Aug 06 00:58:44 dist-test-slave-sb2z krb5kdc[26097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977924, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.252@KRBTEST.COM for kudu/127.27.77.253@KRBTEST.COM
I20260806 00:58:44.663703 26210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.653943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43632 (local address 127.27.77.254:46449)
0806 00:58:44.654195 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.654199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.654213 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:58:44.654250 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.654252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.654296 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.654382 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.655181 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.655853 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.656861 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.657144 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.657266 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.660029 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:44.660032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:44.660034 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:44.660056 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:44.661772 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.662287 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.662290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.662291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.662364 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.662679 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.662683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.662684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.662912 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:44.663122 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:44.663341 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 00:58:44.663469 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":101,"threads_started":1}
I20260806 00:58:44.664503 26411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.655796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.253:46329 (local address 127.0.0.1:53202)
0806 00:58:44.655955 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.655969 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:44.656037 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:44.656300 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:44.656303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:44.656472 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:44.656565 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.656577 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.657287 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.657292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.657965 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.657974 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.658469 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.658473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.658579 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.659023 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:44.659035 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:44.660461 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:44.662680 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.662683 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.662685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.662914 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.663261 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.663267 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.663270 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.663364 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.663947 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:44.663950 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:44.664061 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 00:58:44.664188 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 00:58:44.664333 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 00:58:44.664543 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.655933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53202 (local address 127.27.77.253:46329)
0806 00:58:44.656079 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.656085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.656101 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:58:44.656144 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.656147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.656195 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.656286 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.656694 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.657177 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.658103 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.658349 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.658457 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.660617 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:44.660621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:44.660624 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:44.660657 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:44.662519 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.663025 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.663030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.663032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.663137 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.663480 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.663484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.663485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.663770 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:44.663894 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:44.664250 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 00:58:44.664386 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:58:44.665421 26405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:58:44 dist-test-slave-sb2z krb5kdc[26097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.77.253@KRBTEST.COM
Aug 06 00:58:44 dist-test-slave-sb2z krb5kdc[26097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977924, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.252@KRBTEST.COM for kudu/127.27.77.252@KRBTEST.COM
I20260806 00:58:44.674916 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.665241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53204 (local address 127.27.77.253:46329)
0806 00:58:44.665363 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.665367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.665390 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:58:44.665517 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.665520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.665560 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.665627 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.666369 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.667036 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.668210 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.668437 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.668554 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.671388 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:44.671394 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:44.671400 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:44.671428 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:44.673210 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.673647 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.673653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.673658 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.673748 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.674046 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.674051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.674054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.674254 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:44.674506 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:44.674609 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 00:58:44.674737 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:58:44.674916 26411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.665999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58910)
0806 00:58:44.666211 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.666221 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:44.666293 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:44.666482 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:44.666484 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:44.666623 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:44.666697 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.666702 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.668174 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.668177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.668729 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.668738 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.669265 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.669268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.669393 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.669908 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:44.669920 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:44.671706 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:44.673360 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.673364 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.673365 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.673568 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.673891 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.673893 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.673895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.673977 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.674403 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:44.674406 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:44.674505 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 00:58:44.674624 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 00:58:44.674746 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":24}
I20260806 00:58:44.675046 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.666061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58910 (local address 127.27.77.252:42393)
0806 00:58:44.666174 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.666181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.666289 (+ 108us) server_negotiation.cc:407] Connection header received
0806 00:58:44.666358 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.666361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.666405 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.666478 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.666803 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.668049 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.668862 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.669139 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.669262 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.671839 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:44.671843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:44.671845 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:44.671868 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:44.673221 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.673694 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.673697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.673698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.673768 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.674102 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.674105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.674106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.674273 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:44.674386 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:44.674705 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 00:58:44.674825 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":30}
I20260806 00:58:44.678952 26405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587: Bootstrap starting.
I20260806 00:58:44.679262 27959 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:58:44.680593 26167 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4607888aa08f430ab0ffec961689edaa"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43632
I20260806 00:58:44.682024 26405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:44.682811 26405 log.cc:824] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:44.685047 26405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587: No bootstrap required, opened a new log
I20260806 00:58:44.689368 26405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } }
I20260806 00:58:44.689658 26405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b32a902e36b433f844bd36a97035587, State: Initialized, Role: FOLLOWER
I20260806 00:58:44.690014 26405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } }
I20260806 00:58:44.690559 26415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } } }
I20260806 00:58:44.690697 26415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:58:44.691471 26405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 00:58:44.697114 26430 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:58:44.697214 26430 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:58:44.695559 26317 master_runner.cc:186] Error getting master registration for 127.27.77.254:46449: OK, instance_id { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" instance_seqno: 1785977924134060 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 00:58:44.698553 26317 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:58:44.772506 26307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } attempt: 4
Aug 06 00:58:44 dist-test-slave-sb2z krb5kdc[26097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977924, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.253@KRBTEST.COM for kudu/127.27.77.252@KRBTEST.COM
I20260806 00:58:44.778926 26205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } attempt: 4
I20260806 00:58:44.782825 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.773220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58912)
0806 00:58:44.773383 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.773403 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:44.773515 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:44.773800 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:44.773803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:44.774032 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:44.774131 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.774137 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.774921 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.774926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.775469 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.775477 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.776027 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.776031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.776124 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.776705 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:44.776718 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:44.778519 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:44.780601 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.780605 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.780607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.780867 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.781523 (+ 656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.781526 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.781528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.781632 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.782110 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:44.782113 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:44.782234 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 00:58:44.782344 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 00:58:44.782570 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 00:58:44.783807 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.773294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58912 (local address 127.27.77.252:42393)
0806 00:58:44.773483 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.773488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.773506 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:58:44.773620 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.773623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.773673 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.773771 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.774271 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.774797 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.775650 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.775890 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.776008 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.778694 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:44.778698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:44.778701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:44.778733 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:44.780463 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.780976 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.780984 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.780988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.781085 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.781713 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.781717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.781718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.781998 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:44.782100 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:44.783464 (+ 1364us) server_negotiation.cc:299] Negotiation successful
0806 00:58:44.783633 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
Aug 06 00:58:44 dist-test-slave-sb2z krb5kdc[26097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977924, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.77.254@KRBTEST.COM for kudu/127.27.77.252@KRBTEST.COM
I20260806 00:58:44.787024 26307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2: Bootstrap starting.
I20260806 00:58:44.789950 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.779998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.77.252:42393 (local address 127.0.0.1:58924)
0806 00:58:44.780324 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 00:58:44.780339 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:58:44.780423 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:58:44.781084 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:58:44.781087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:58:44.781273 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:58:44.781384 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.781391 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.782356 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.782360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.783142 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:58:44.783155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.783769 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.783773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:58:44.783864 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.784392 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:58:44.784407 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:58:44.785940 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:58:44.788231 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.788235 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.788237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.788508 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.788829 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:58:44.788832 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:58:44.788834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:58:44.788918 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:58:44.789346 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:58:44.789349 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:58:44.789465 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 00:58:44.789595 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 00:58:44.789757 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235}
I20260806 00:58:44.789995 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.780313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58924 (local address 127.27.77.252:42393)
0806 00:58:44.780834 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.780842 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.780855 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:58:44.780890 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.780893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.780954 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.781057 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.781504 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.782247 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.783253 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.783636 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.783776 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.786101 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:44.786105 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:44.786108 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:44.786134 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:44.787980 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.788627 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.788630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.788632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.788738 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.789010 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.789013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.789015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.789257 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:44.789359 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:44.789684 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 00:58:44.789822 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":172,"threads_started":1}
I20260806 00:58:44.790421 26307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:44.791476 26307 log.cc:824] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:44.793715 26205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa: Bootstrap starting.
I20260806 00:58:44.793937 26307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2: No bootstrap required, opened a new log
I20260806 00:58:44.796850 26205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa: Neither blocks nor log segments found. Creating new log.
I20260806 00:58:44.797650 26307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } }
I20260806 00:58:44.797782 26205 log.cc:824] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa: Log is configured to *not* fsync() on all Append() calls
I20260806 00:58:44.797888 26307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 574ea8d990cf4fcbb6ea04737d3a56b2, State: Initialized, Role: FOLLOWER
I20260806 00:58:44.798233 26307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } }
I20260806 00:58:44.799513 26307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:44.799836 26435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } } }
I20260806 00:58:44.800043 26435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:58:44.800161 26205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa: No bootstrap required, opened a new log
I20260806 00:58:44.804188 26205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } }
I20260806 00:58:44.804502 26205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4607888aa08f430ab0ffec961689edaa, State: Initialized, Role: FOLLOWER
I20260806 00:58:44.804895 26205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } }
I20260806 00:58:44.805586 26450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } } }
I20260806 00:58:44.805790 26450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:58:44.806456 26205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:58:44.806528 26167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 00:58:44.806658 26167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:44.803385 26215 master_runner.cc:186] Error getting master registration for 127.27.77.254:46449: OK, instance_id { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" instance_seqno: 1785977924134060 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 00:58:44.807613 26215 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:58:44.808146 26167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } }
I20260806 00:58:44.809669 26275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4607888aa08f430ab0ffec961689edaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2"
I20260806 00:58:44.809741 26366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4607888aa08f430ab0ffec961689edaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5b32a902e36b433f844bd36a97035587"
I20260806 00:58:44.809919 26275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:44.809950 26366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:58:44.811317 26366 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4607888aa08f430ab0ffec961689edaa in term 1.
I20260806 00:58:44.811317 26275 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4607888aa08f430ab0ffec961689edaa in term 1.
I20260806 00:58:44.809109 26167 leader_election.cc:288] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [CANDIDATE]: Term 1 election: Requested vote from peers 574ea8d990cf4fcbb6ea04737d3a56b2 (127.27.77.253:46329), 5b32a902e36b433f844bd36a97035587 (127.27.77.252:42393)
I20260806 00:58:44.812294 26137 leader_election.cc:302] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4607888aa08f430ab0ffec961689edaa, 5b32a902e36b433f844bd36a97035587; no voters:
W20260806 00:58:44.813638 26449 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:58:44.813711 26449 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 00:58:44.814715 26470 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:58:44.814851 26470 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:58:44.818722 26116 master_runner.cc:263] No leader master found from master 4607888aa08f430ab0ffec961689edaa
I20260806 00:58:44.818805 26116 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:58:44.818904 26450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:58:44.819142 26450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [term 1 LEADER]: Becoming Leader. State: Replica: 4607888aa08f430ab0ffec961689edaa, State: Running, Role: LEADER
I20260806 00:58:44.819589 26450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } }
I20260806 00:58:44.821926 26450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4607888aa08f430ab0ffec961689edaa. Latest consensus state: current_term: 1 leader_uuid: "4607888aa08f430ab0ffec961689edaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } } }
I20260806 00:58:44.822062 26450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:44.822459 26483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:58:44.823477 26483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:58:44.831099 26366 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587 [term 1 FOLLOWER]: Refusing update from remote peer 4607888aa08f430ab0ffec961689edaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:58:44.831900 26450 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:58:44.836201 26415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4607888aa08f430ab0ffec961689edaa. Latest consensus state: current_term: 1 leader_uuid: "4607888aa08f430ab0ffec961689edaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } } }
I20260806 00:58:44.836349 26415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:58:44.842303 26275 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2 [term 1 FOLLOWER]: Refusing update from remote peer 4607888aa08f430ab0ffec961689edaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:58:44.843092 26450 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:58:44.847506 26435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4607888aa08f430ab0ffec961689edaa. Latest consensus state: current_term: 1 leader_uuid: "4607888aa08f430ab0ffec961689edaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } } }
I20260806 00:58:44.847708 26435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:58:44.853906 26450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4607888aa08f430ab0ffec961689edaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } } }
I20260806 00:58:44.854094 26450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:44.855386 26435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4607888aa08f430ab0ffec961689edaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } } }
I20260806 00:58:44.855546 26435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:58:44.855962 26210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.819434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43634 (local address 127.27.77.254:46449)
0806 00:58:44.819574 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.819579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.823819 (+ 4240us) server_negotiation.cc:407] Connection header received
0806 00:58:44.824021 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.824025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.824077 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.824135 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.833355 (+ 9220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.834173 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.838934 (+ 4761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.839291 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.839370 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.846834 (+ 7464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:44.846841 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:44.846847 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:44.846880 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:44.848632 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.850245 (+ 1613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.850249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.850251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.850348 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.852956 (+ 2608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.852960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.852962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.853211 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:44.853326 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:44.853690 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 00:58:44.853826 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:58:44.857131 26488 mvcc.cc:205] Tried to move back new op lower bound from 7315365580099289088 to 7315365580066471936. Current Snapshot: MvccSnapshot[applied={T|T < 7315365580099289088}]
I20260806 00:58:44.857648 26491 mvcc.cc:205] Tried to move back new op lower bound from 7315365580099289088 to 7315365580066471936. Current Snapshot: MvccSnapshot[applied={T|T < 7315365580099289088}]
I20260806 00:58:44.858204 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.819874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53218 (local address 127.27.77.253:46329)
0806 00:58:44.819996 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.820001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.831925 (+ 11924us) server_negotiation.cc:407] Connection header received
0806 00:58:44.832147 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.832150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.832212 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.832266 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.836976 (+ 4710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.837767 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.843393 (+ 5626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.843782 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.843860 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.850709 (+ 6849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:44.850714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:44.850717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:44.850747 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:44.852438 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.854257 (+ 1819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.854261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.854263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.854368 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.854795 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.854799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.854801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.855024 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:44.855147 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:44.857921 (+ 2774us) server_negotiation.cc:299] Negotiation successful
0806 00:58:44.858051 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:58:44.860430 26450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4607888aa08f430ab0ffec961689edaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } } }
I20260806 00:58:44.860562 26450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa [sys.catalog]: This master's current role is: LEADER
I20260806 00:58:44.861236 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:44.820490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58928 (local address 127.27.77.252:42393)
0806 00:58:44.820628 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.820632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.835636 (+ 15004us) server_negotiation.cc:407] Connection header received
0806 00:58:44.835806 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.835810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.835865 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.835916 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.840294 (+ 4378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.841128 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.845665 (+ 4537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:44.846050 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:44.846134 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:44.855493 (+ 9359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:58:44.855497 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:58:44.855501 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:58:44.855583 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:58:44.857348 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.859665 (+ 2317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.859669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.859671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.859771 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:58:44.860144 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:58:44.860148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:58:44.860150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:58:44.860381 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:58:44.860516 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:44.860900 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 00:58:44.861074 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:58:44.861842 26415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4607888aa08f430ab0ffec961689edaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4607888aa08f430ab0ffec961689edaa" member_type: VOTER last_known_addr { host: "127.27.77.254" port: 46449 } } peers { permanent_uuid: "574ea8d990cf4fcbb6ea04737d3a56b2" member_type: VOTER last_known_addr { host: "127.27.77.253" port: 46329 } } peers { permanent_uuid: "5b32a902e36b433f844bd36a97035587" member_type: VOTER last_known_addr { host: "127.27.77.252" port: 42393 } } }
I20260806 00:58:44.861990 26415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:58:44.865613 26489 mvcc.cc:205] Tried to move back new op lower bound from 7315365580099289088 to 7315365580066471936. Current Snapshot: MvccSnapshot[applied={T|T < 7315365580099289088}]
I20260806 00:58:44.867813 26483 catalog_manager.cc:1347] Generated new cluster ID: e6f5c9a65a9d45c2b05c99c77d557ade
I20260806 00:58:44.867923 26483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:58:44.912014 26483 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:58:44.913461 26483 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:58:44.926848 26483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4607888aa08f430ab0ffec961689edaa: Generated new TSK 0
I20260806 00:58:44.927814 26483 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:58:44.960369 26210 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:44.957904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43642 (local address 127.27.77.254:46449)
0806 00:58:44.958055 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.958059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.959096 (+ 1037us) server_negotiation.cc:407] Connection header received
0806 00:58:44.959329 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.959333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.959388 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.959442 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.960240 (+ 798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43642: BlockingRecv error: recv got EOF from 127.0.0.1:43642 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 00:58:44.960578 26312 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:44.957879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53220 (local address 127.27.77.253:46329)
0806 00:58:44.958016 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.958021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.959092 (+ 1071us) server_negotiation.cc:407] Connection header received
0806 00:58:44.959333 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.959337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.959394 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.959451 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.960446 (+ 995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53220: BlockingRecv error: recv got EOF from 127.0.0.1:53220 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260806 00:58:44.962117 26432 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:44.958728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58932 (local address 127.27.77.252:42393)
0806 00:58:44.958852 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.958856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.960175 (+ 1319us) server_negotiation.cc:407] Connection header received
0806 00:58:44.961187 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.961192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.961251 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.961309 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.961982 (+ 673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58932: BlockingRecv error: recv got EOF from 127.0.0.1:58932 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 00:58:44.965651 26312 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:44.964352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53230 (local address 127.27.77.253:46329)
0806 00:58:44.964474 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.964478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.964494 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:58:44.964734 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.964737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.964820 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.964918 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.965539 (+ 621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53230: BlockingRecv error: recv got EOF from 127.0.0.1:53230 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 00:58:44.969800 26312 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:44.968824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53242 (local address 127.27.77.253:46329)
0806 00:58:44.968935 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.968943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.968959 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:58:44.969012 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.969015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.969066 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.969163 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.969683 (+ 520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53242: BlockingRecv error: recv got EOF from 127.0.0.1:53242 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 00:58:44.975643 26312 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:44.974248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53254 (local address 127.27.77.253:46329)
0806 00:58:44.974356 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.974360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.974630 (+ 270us) server_negotiation.cc:407] Connection header received
0806 00:58:44.974772 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.974776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.974849 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.974946 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.975468 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53254: BlockingRecv error: recv got EOF from 127.0.0.1:53254 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 00:58:44.983289 26312 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:44.981798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53266 (local address 127.27.77.253:46329)
0806 00:58:44.981941 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.981946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.982212 (+ 266us) server_negotiation.cc:407] Connection header received
0806 00:58:44.982386 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.982390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.982457 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.982552 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.983161 (+ 609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53266: BlockingRecv error: recv got EOF from 127.0.0.1:53266 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 00:58:44.995432 26312 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:44.994003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53272 (local address 127.27.77.253:46329)
0806 00:58:44.994137 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:44.994142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:44.994402 (+ 260us) server_negotiation.cc:407] Connection header received
0806 00:58:44.994546 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:44.994550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:44.994627 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:44.994737 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:44.995289 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53272: BlockingRecv error: recv got EOF from 127.0.0.1:53272 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 00:58:45.015405 26312 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:45.014091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53286 (local address 127.27.77.253:46329)
0806 00:58:45.014246 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:45.014251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:45.014440 (+ 189us) server_negotiation.cc:407] Connection header received
0806 00:58:45.014560 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:45.014564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:45.014629 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:45.014729 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:45.015269 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53286: BlockingRecv error: recv got EOF from 127.0.0.1:53286 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 00:58:45.051944 26312 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:45.050484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53290 (local address 127.27.77.253:46329)
0806 00:58:45.050723 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:45.050731 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:45.050838 (+ 107us) server_negotiation.cc:407] Connection header received
0806 00:58:45.051034 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:45.051040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:45.051132 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:45.051240 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:45.051792 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53290: BlockingRecv error: recv got EOF from 127.0.0.1:53290 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
W20260806 00:58:45.120103 26312 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:45.118706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53294 (local address 127.27.77.253:46329)
0806 00:58:45.118884 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:45.118890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:45.119064 (+ 174us) server_negotiation.cc:407] Connection header received
0806 00:58:45.119289 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:45.119292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:45.119344 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:45.119438 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:45.119971 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53294: BlockingRecv error: recv got EOF from 127.0.0.1:53294 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 00:58:45.252132 26312 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:45.250698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53296 (local address 127.27.77.253:46329)
0806 00:58:45.250898 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:45.250903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:45.251057 (+ 154us) server_negotiation.cc:407] Connection header received
0806 00:58:45.251295 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:45.251301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:45.251356 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:45.251453 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:45.251989 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53296: BlockingRecv error: recv got EOF from 127.0.0.1:53296 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260806 00:58:45.511967 26312 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:58:45.510639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53300 (local address 127.27.77.253:46329)
0806 00:58:45.510812 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:45.510821 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:45.510887 (+ 66us) server_negotiation.cc:407] Connection header received
0806 00:58:45.511082 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:45.511086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:45.511144 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:45.511241 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:58:45.511808 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53300: BlockingRecv error: recv got EOF from 127.0.0.1:53300 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 00:58:45.698673 26430 catalog_manager.cc:1259] Loaded cluster ID: e6f5c9a65a9d45c2b05c99c77d557ade
I20260806 00:58:45.698765 26430 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587: loading cluster ID for follower catalog manager: success
I20260806 00:58:45.700806 26430 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587: acquiring CA information for follower catalog manager: success
I20260806 00:58:45.702113 26430 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5b32a902e36b433f844bd36a97035587: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:58:45.815259 26449 catalog_manager.cc:1259] Loaded cluster ID: e6f5c9a65a9d45c2b05c99c77d557ade
I20260806 00:58:45.815398 26449 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2: loading cluster ID for follower catalog manager: success
I20260806 00:58:45.817266 26449 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2: acquiring CA information for follower catalog manager: success
I20260806 00:58:45.818503 26449 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 574ea8d990cf4fcbb6ea04737d3a56b2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:58:46.030987 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:46.026543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53306 (local address 127.27.77.253:46329)
0806 00:58:46.026815 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:46.026820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:46.026838 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:58:46.026936 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:46.026938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:46.027077 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:46.027189 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:58:46.027555 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:46.028190 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:46.029216 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:46.029453 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:46.029550 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:46.029933 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:58:46.030432 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:58:46.030534 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:46.030677 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 00:58:46.030740 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":75,"threads_started":1}
I20260806 00:58:46.038847 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:46.034572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58940 (local address 127.27.77.252:42393)
0806 00:58:46.034835 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:46.034839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:46.034853 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:58:46.034943 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:46.034946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:46.035066 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:46.035172 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:58:46.035448 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:46.036020 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:46.036874 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:46.037096 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:46.037184 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:46.037580 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:58:46.038094 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:58:46.038221 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:46.038428 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 00:58:46.038627 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":112,"threads_started":1}
I20260806 00:58:46.047585 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:46.042472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53314 (local address 127.27.77.253:46329)
0806 00:58:46.042749 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:46.042754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:46.043036 (+ 282us) server_negotiation.cc:407] Connection header received
0806 00:58:46.043163 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:46.043166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:46.043224 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:46.043304 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:58:46.043768 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:46.044372 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:46.046307 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:46.046644 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:46.046743 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:46.047004 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:58:46.047095 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:58:46.047148 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:46.047322 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 00:58:46.047380 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260806 00:58:46.047557 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:46.042774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58952 (local address 127.27.77.252:42393)
0806 00:58:46.042917 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:46.042921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:46.043278 (+ 357us) server_negotiation.cc:407] Connection header received
0806 00:58:46.043428 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:46.043431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:46.043499 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:46.043577 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:58:46.043979 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:46.044728 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:46.045870 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:46.046226 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:46.046334 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:46.046675 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:58:46.046766 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:58:46.046843 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:46.046943 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 00:58:46.047255 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 00:58:46.052004 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:58:46.042473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43652 (local address 127.27.77.254:46449)
0806 00:58:46.042754 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 00:58:46.042761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:58:46.042775 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:58:46.042991 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:58:46.043015 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:58:46.043234 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:58:46.043352 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:58:46.043636 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:46.044539 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:46.045963 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:58:46.046343 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:58:46.046443 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:58:46.051085 (+ 4642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:58:46.051425 (+ 340us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:58:46.051499 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 00:58:46.051704 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 00:58:46.051806 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":87,"threads_started":1}
W20260806 00:58:46.973168 27959 net_util.cc:504] Time spent look up canonical hostname for localhost 'dist-test-slave-sb2z': real 0.920s user 0.000s sys 0.000s
I20260806 00:58:46.976402 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 26116
I20260806 00:58:46.995586 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 26215
I20260806 00:58:47.011154 27959 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxB88Ks/build/debug/bin/kudu with pid 26317
2026-08-06T00:58:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4451 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (7887 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (307961 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST